UP | HOME

PostgreSQL

1. What is an schema

An schema is basically a logical segmentation that holds database objects like views, indexes, data types, functions, store procedures and operators. here is a good post about this in detail.

2. Indexes

B-Tree is the default that you get when you do CREATE INDEX. Virtually all databases have some B-tree indexes. B-tree indexes are optimized for when a row has a single key value.

More on database indexing:

Indexing is a database optimization technique used to speed up data retrieval operations on tables. In PostgreSQL, indexes are created on one or more columns of a table to improve query performance. They work like a table of contents in a book, allowing the database to quickly locate rows based on the indexed columns.

Here are some common types of indexes in PostgreSQL:

  • B-tree Index: Suitable for equality and range queries.
  • Unique Index: Ensures that the indexed column(s) contain unique values.
  • Partial Index: Indexes a subset of rows based on a condition.
  • GIN (Generalized Inverted Index): Used for full-text search and array types.
  • GiST (Generalized Search Tree): Used for complex data types like geometric or text search.
  • SP-GiST (Space-Partitioned Generalized Search Tree): Suitable for multidimensional data.
  • BRIN (Block Range INdex): Efficient for large tables with sorted data.

How do we create an index in PostgreSQL:

-- stat.rfq_products table:
-- If you frequently query this table based on the rfq_id column, which references the stat.request_for_quotes table, you can create an index on this column for faster lookups.
CREATE INDEX idx_rfq_products_rfq_id ON stat.rfq_products (rfq_id);
  • Postgres automatically creates an index for the primary key of every table.
  • Postgres automatically creates an index for any 'unique' constraint.

These don't get listed under indexes in PGAdmin!

We can list all the existing indexes in a postgres database with this query:

SELECT relname, relkind FROM pg_class -- pg_class lists all the kind of objects in the database
       WHERE relkind = 'i'; -- this tells to retrieve only the kind of 'i' which stands for _index_

2.1. Downsides of Indexes

  • Can be large! Stores data from at least one column of the real table
  • Slows down insert/update/delete - the index has to be updated!
  • Index might not actually get used!

3. Foreign keys

More about foreign keys:

  • postgres official page

4. Postgraphile Notes

5. Data types

Check out this post.

JSON data type stackoverflow overview comment.

raw JSON vs JSON Binary:

If you know before hand that you will not be performing JSON querying operations, then use the JSON data type. For all other cases, use JSONB. More here.

6. Database design process

  1. What kind of thing are we storing?
  2. what properties does this thing have?
  3. what type of data does each of those properties contain?

7. Notes from PostgreSQL official manual

  • Tables are grouped into databases, and a collection of databases managed by a single PostgreSQL server instance constitutes a database cluster

8. Common Table Expressions

Abbreviated as CTE is a one-time result set that only exists for the duration of the query. It allows us to refer to data within a single SELECT, INSERT, UPDATE, DELETE, CREATE VIEW, or MERGE statement's execution scope.

9. Recursive queries in SQL

To write a recursive query in SQL we need to use a CTE. The syntax is the following:

20230410_153812_UAEctx.png

So for instance if we like to print a sequence of numbers using this recursive approach we could write the following sql:

with recursive numbers AS (
        select 1 as n
          union all
        select n + 1 from numbers where n < 5
)

select * from numbers;
n
1
2
3
4
5

10. Snippets Examples

insert into cities(name, location) values ('Rio bravo', point(100, 100));
INSERT 0 1
update cities
  set location = point(200, 200)
  where name = 'Rio bravo'
  returning *; -- returns updated rows
name	location
Rio bravo	(200,200)
UPDATE 1
delete from cities
where name ilike 'Rio Bravo'; -- case insensitive

#+RESULTS

DELETE 2
select * from cities;
name	location
reynosa	(129,120)
monterrey	(102,30)
Rio bravo	(200,200)
BEGIN;
UPDATE accounts SET balance = balance - 100.00
 WHERE name = 'Alice';
-- etc etc
COMMIT;
-- If, partway through the transaction, we decide we do not want to commit (perhaps we just noticed that
-- Alice's balance went negative), we can issue the command ROLLBACK instead of COMMIT, and all our
-- updates so far will be canceled.





-- It's possible to control the statements in a transaction in a more granular fashion through the use of save-
-- points. Savepoints allow you to selectively discard parts of the transaction, while committing the rest. After
-- defining a savepoint with SAVEPOINT, you can if needed roll back to the savepoint with ROLLBACK TO.
-- All the transaction's database changes between defining the savepoint and rolling back to it are discarded,
-- but changes earlier than the savepoint are kept.
-- Remembering the bank database, suppose we debit $100.00 from Alice's account, and credit Bob's account,
-- only to find later that we should have credited Wally's account. We could do it using savepoints like this:
BEGIN;
UPDATE accounts SET balance = balance - 100.00
 WHERE name = 'Alice';
SAVEPOINT my_savepoint;
UPDATE accounts SET balance = balance + 100.00
 WHERE name = 'Bob';
-- oops ... forget that and use Wally's account
ROLLBACK TO my_savepoint;
UPDATE accounts SET balance = balance + 100.00
 WHERE name = 'Wally';
COMMIT;

create view myView as
       select * from weather where city = 'reynosa';

select * from myView;
-- default role in this session is 'postgres'
create role role_x; -- creating role with name role_x
select current_role; -- this outputs 'postgres'
set role role_x; -- set role x as the current role
select current_role; -- outputs 'role_x'
CREATE ROLE
current_role
postgres
SET
current_role
role_x
-- this below function just returns a text 
create or replace function public.some_function()
 returns text
    language plpgsql
as $$
    begin
        return 'Hello World';
    end;
$$;


-- executing the function
select some_function(); -- this returns 'Hello World'
CREATE FUNCTION
some_function
Hello World
-- Now lets restrict the execution of 'some_function' function to an specific roke
-- in this case, we will prevent 'role_x' to execute 'some_function' function.

-- let's change the role
set role role_x;

-- now let's execute the function
select some_function(); -- this will output 'Hello World'

-- let's revoke the permission of execution to role x
-- to achieve this, first let's change the role to a role that
-- has enough permissions to do this, like a superuser role, like 'postgres'
-- which is the default role in PostgreSQL
-- @Note: postgres role is a superuser role created by default during the
-- installation of PostgreSQL. The postgres role has full privileges over
-- the entire database system and can perform any operation, including granting
-- and revoking privileges for other roles. It's similar to the root user in a Unix system.
set role postgres;
revoke execute on function some_function() from role_x;

-- lets verify trying to execute the function from role_x
set role role_x;
select some_function(); -- this still not prevents the execution because this role inherits from 'public' role
-- so let's revoke really the function.
set role postgres;
revoke execute on function some_function() from public;
set role role_x;

DO $$
DECLARE
  result text;
BEGIN
    BEGIN
        result := select some_function();
        RAISE NOTICE 'Function returned: %', result;
    EXCEPTION WHEN INSUFFICIENT_PRIVILEGE THEN
        RAISE WARNING 'An error occurred: %', SQLERRM;
    END;
END;
$$ LANGUAGE plpgsql;

SET

11. Data Access Control with RLS

RLS stands for Row Level Security.

This pdf contains a good summary about row level security access in PostgreSQL. Check out: https://learn.graphile.org/docs/PostgreSQL_Row_Level_Security_Infosheet.pdf

12. Course notes

12.1. Migrations

Big Lessons:

  1. Changes to the DB structure and changes to clients need to be made at precisely the same time.
  2. When working with other engineers, we need a really easy way to tie the structure of our database to our code.

12.2. Foreign keys


12.3. Some util queries

-- Get size in kb of certain table
SELECT pg_size_pretty(pg_relation_size('TABLE_NAME or INDEX_NAME'))


-- Explain analyze
-- if we add this two words before an sql statement we can see the planning and metrics (time in ms) that a postgreSQL operation took.
EXPLAIN ANALYZE select * from users;

13. Random notes

13.1. Updating DB Data after a migration

Instead of updating user data inside a migration, what we can do is build an sql script with the statements wrapped inside a transaction. Doing it so, we can prevent having unsync or corrupted data.

We can run the script to update the data with psql like so:

   psql -U postgres -h localhost -p 5433 -d e2e -v ON_ERROR_STOP=1 <<EOF
BEGIN;
\i migration_163_rbac_updates.sql
COMMIT;
EOF

e.g. migration_163_rbac_updates.sql:

-- set stat_admin (employee) users default admin_role permission
DO $$
DECLARE
    v_admin_role_id integer;
BEGIN
    SELECT id INTO v_admin_role_id FROM stat.roles WHERE internal_name = 'admin_role';
    UPDATE stat.users
    SET assigned_role_id = v_admin_role_id
    WHERE role = 'stat_admin' or role = 'stat_super_admin';
END $$;


-- puedes forzar un error para confirmar que si ocurre algún error entonces se hace un rolllback automatico.
-- Mira mas detalles de la variable que pasamos al comando de psql: -v ON_ERROR_STOP=1
-- DO $$
-- BEGIN
--   RAISE EXCEPTION 'Error forzado para pruebas';
-- END $$;

-- set stat_user (client) users default client role permission
DO $$
DECLARE
    v_client_role_id integer;
BEGIN
    SELECT id INTO v_client_role_id FROM stat.roles WHERE internal_name = 'client_role';
    UPDATE stat.users
    SET assigned_role_id = v_client_role_id
    WHERE role = 'stat_user';
END $$;

-- update to super admin the required users for e2e and integration tests
update stat.users u
set role = 'stat_super_admin' where u.email = 'e2eadminkbi@e2e.com' or u.email = 'e2eadminkbi@kbibiopharma.com';