In this post, I am sharing an example of applying Full Text Search on PostgreSQL Table with Index.

Here, I am using tsvector for full text search which is document type and uses match operator like @@.

PostgreSQL: Example of Trigram Index for Full Text Search using pg_trgm Extension

Below is an example:

Create a sample table with data:

CREATE TABLE tbl_Textdata
(id int, txt character varying);

INSERT INTO tbl_Textdata VALUES
(1,'I am DBA'),(2,'database research & development'),(3,'think for data only')
,(4,'data is important'),(5,'data is a future'),(6,'I am data owner');

Do Full Text Searching:

SELECT txt
FROM tbl_Textdata
WHERE to_tsvector(txt) @@ to_tsquery('data');

Result:

txt
--------------------------
'think for data only'
'data is important'
'data is a future'
'I am data owner'

Apply an Index on require Text Pattern:

CREATE INDEX tbl_Textdata_txt_idx ON tbl_Textdata USING GIN (to_tsvector('english',txt));

Check the execution plan:

explain analyze
SELECT txt
FROM tbl_Textdata
WHERE to_tsvector(txt) @@ to_tsquery('data');

'Seq Scan on tbl_textdata  (cost=0.00..4.07 rows=1 width=32) (actual time=0.043..0.058 rows=4 loops=1)'
'  Filter: (to_tsvector((txt)::text) @@ to_tsquery('data'::text))'
'  Rows Removed by Filter: 2'
'Planning time: 5.494 ms'
'Execution time: 0.083 ms'

Important notice: Never try in your Production or any other Live server. This post is only ad-hoc and test server.

Data Durability is an important part of Database ACID property. In this post, I am sharing few parameters which we can disable for achieving good performance of PostgreSQL Database Server.

We should disable Non-Durable parameters on our test server where we need only POC related work, and if your system doesn’t require any guarantee of data, then you can disable.

Why disabling the Non-Durable parameters?

PostgreSQL guarantees the full durability of data even if the server crashes or loses power.
But committing every single transaction in datafile and log file is also extra overhead on the database server.
So simple, just disable that parameter for achieving good performance.

ACID Properties in Database System (Atomicity, Consistency, Isolation, Durability)

Below are parameters:

fsync: Turn off – no need to flush data to disk

synchronous_commit: Turn off – not need to write to disk on every commit

full_page_writes: Turn off – no need to guard against partial page writes

max_wal_size: Increase – reduces the frequency of checkpoints

checkpoint_timeout: Increase – reduces the frequency of checkpoints

In this post, I am sharing the use of “unaccent” extension of PostgreSQL.

Use unaccent, for removing diacritic signs from the string and convert into the valid character as per the specified collation.

When we are importing data from different types of datasets, we can find diacritic signs in the data. We can use unaccent() for removing those signs.

Check below demonstration:

Create an extension:

CREATE EXTENSION unaccent;

Use unaccent():

SELECT unaccent('ô ã À Æ ß Ä');

Result:

unaccent
------------------
'o a A AE ss A'

In this post, I am sharing few mathematical operators for finding factorial, absolute and negation value in PostgreSQL.

I was reading the official the official page of PostgreSQL and found few interesting operators which we can use for the mathematical purpose.

Check below examples:

! operator for factorial:

SELECT 8 ! AS "8 factorial";

8 factorial
------------
'40320'

@ operator for absolute:

SELECT @ '-8.8' AS "absolute value";

absolute values
------------
'40320'

~ operator for negation:

SELECT ~ CAST('16' AS int8) AS "negation value";

negation value
---------------
'-17'

In this post, I am sharing a script for transforming normal DB user to Super User in PostgreSQL.

The Super User is the admin user and it has all kind of access to PostgreSQL Server. We can create multiple superusers in PostgreSQL.

I need this kind of handy commands/scripts because when any new dba join the team, giving basic permission of database and later the same user requires superuser permission.

Convert a user into Super User:

ALTER USER user_name WITH SUPERUSER;

Revoke Super User Permission:

ALTER USER user_name WITH NOSUPERUSER;

As per the en.wikibooks.org:

Bash is a “Unix shell”: a command-line interface for interacting with the operating system. It has the ability to run an entire script of commands, known as a “Bash shell script”.

I recently started to create UNIX / LINUX Bash Shell script for enhancing my PostgreSQL DBA Work. You can create a Bash shell script and can connect PostgreSQL using psql.

In this post, I am sharing a sample bash shell script to execute psql commands.

Open a new file in vi editor:

[root@dbrnd anvesh]$ vi helloworld.bash

Press [i] key to get an insert prompt:

Copy below sample Bash Shell Script and Paste into vi editor:

#!/bin/bash

###################################################
# First Bash Shell script to execute psql command 
###################################################

#Set the value of variable
database="database_name"

#Execute few psql commands: 
#Note: you can also add -h hostname -U username in the below commands. 

psql -d $database -c "CREATE TABLE public.tbl_students(rno integer, name character varying)"
psql -d $database -c "INSERT INTO public.tbl_students VALUES (1,'Anvesh'),(2,'Neevan'),(3,'Roy'),(4,'Martin'),(5,'Jenny')"
psql -d $database -c "SELECT *FROM public.tbl_students"

#Assign table count to variable
TableCount=$(psql -d $database -t -c "select count(1) from public.tbl_students")

#Print the value of variable
echo "Total table records count....:"$TableCount

Press [Esc] key to get a command prompt and execute below command:
To save and exit vi editor.

:wq

Give permission to the file:

[root@dbrnd anvesh]$ chmod +x helloworld.bash

Execute this file:

[root@dbrnd anvesh]$ ./helloworld.bash

You can find below result:

rno |  name
-----+--------
   1 | Anvesh
   5 | Jenny
   2 | Neevan
   3 | Roy
   4 | Martin
(5 rows)

Total table records count....: 5

In this post, I am sharing an important tip for PostgreSQL DBAs.

I recently started work with a new PostgreSQL Client. They have total 326 databases in their PostgreSQL Server. They are taking full backup for all the database and this process is taking around total 7 hours.

When I looked this, my first step was to find database statistics. I have found 25% of databases are not being updated frequently.

I added one additional check in a backup process like “skip the backup for those databases which have not been changed for any operations (DDL or DML operation)”.
After this check, the backup process is taking around total 5 hours to complete the job.

Using pg_stat_database, you can check the stats of database and can compare the value for tuple columns with the old value. If you found any changes, your database has been changed.

If any of your DBA reset the PostgreSQL database statistics using something like pg_stat_reset, you have to add some more extra logic to understand the difference between statistics.

Below is a sample demonstration:
Create a backup table to store DatabaseStats:

CREATE TABLE public.tbl_DatabaseStats AS
SELECT 
	datname
	,tup_inserted
	,tup_updated
	,tup_deleted	
FROM pg_stat_database;

Perform few DDLs and DMLs actions:

CREATE TABLE public.tbl_Students(rno int, name character varying);
INSERT INTO public.tbl_Students VALUES(1,'Anvesh');
CREATE TABLE public.tbl_Test88(id int);

Compare the result of below two queries:
If you find differences on any of the columns of your Database, your database stats has been changed and you should take the backup of your database.

SELECT 
	datname
	,tup_inserted
	,tup_updated
	,tup_deleted	
FROM pg_stat_database;

SELECT *FROM public.tbl_DatabaseStats;

While creating a function, You can define the attributes like IMMUTABLE, STABLE, VOLATILE and COST. These attributes inform the query optimizer about the behaviour of the function.

IMMUTABLE: It indicates that the function cannot modify the database and always returns the same result when given the same argument values; that is, it does not do database lookups or otherwise use information not directly present in its argument list.

STABLE: It indicates that the function cannot modify the database, and that within a single table scan it will consistently return the same result for the same argument values, but that its result could change across SQL statements. This is the appropriate selection for functions whose results depend on database lookups, parameter variables (such as the current time zone), etc.

VOLATILE: It indicates that the function value can change even within a single table scan so that no optimizations can make.

COST: It declares the cost per row of the result, which is used by the query planner to find the cheapest plan. The default is COST 100. COST measured in CPU cycles. A higher COST number means more costly. A COST will give the planner a better idea of which strategy to use.

CREATE FUNCTION fn_test()
  RETURNS as
$$
declare
	-- declare variables...
begin	
	-- do operations...	
end
$$ language plpgsql VOLATILE
COST 10000;

This is an important article for PostgreSQL Developer because many of the database developers don’t know about the PostgreSQL Function Volatility.

Every function has a volatility classification, with the possibilities being VOLATILE, STABLE, or IMMUTABLE. VOLATILE is the default.

In this post, I am not going to write 1000 words on it. Because I believe in practical so sharing a good practical example here.
But you must read about the PostgreSQL Function Volatility from this official page.

Check the below practical example and you will get to know that how VOLATILE function is choosing the bad Query Execution Plan.

Create a sample function:

CREATE OR REPLACE FUNCTION fn_getvalues(int, int)
RETURNS int
AS
$$
  BEGIN
       RETURN 
       	CASE WHEN $1 = $2 THEN $1 ELSE $2 END;
  END;
$$ LANGUAGE 'plpgsql';

Check the result of sample function:

SELECT * FROM fn_getvalues(50,80);

fn_getvalues
-----------------
80

Create a sample table with data:

CREATE TABLE tbl_ItemTransactions 
 (
     TranID SERIAL
     ,TransactionDate TIMESTAMPTZ
     ,TransactionName TEXT
 );

CREATE INDEX idx_TranID ON tbl_ItemTransactions (TranID);
 
INSERT INTO tbl_ItemTransactions 
(TransactionDate, TransactionName)
SELECT x, 'dbrnd' 
FROM generate_series('2017-01-01 00:00:00'::timestamptz, '2018-01-31 00:00:00'::timestamptz,'50 seconds'::interval) a(x);

Check the execution plan for below query:
You can find ‘Index Scan’ because the huge table and index applied properly.

EXPLAIN
SELECT *FROM tbl_ItemTransactions
WHERE TranID = 80

'Index Scan using idx_tranid on tbl_itemtransactions  (cost=0.42..8.44 rows=1 width=18)'
'  Index Cond: (tranid = 80)'

Now, Use the early created function in the same query:
You can find ‘Seq Scan’ because the function of the filter is creating a problem. Because VOLATILE is the default volatility classification.

EXPLAIN
SELECT *FROM tbl_ItemTransactions
WHERE TranID = fn_getvalues (50,80);

'Seq Scan on tbl_itemtransactions  (cost=0.00..183520.26 rows=1 width=18)'
'  Filter: (tranid = fn_getvalues(50, 80))'

Now, Let’s create an IMMUTABLE function:

CREATE OR REPLACE FUNCTION fn_getvalues(int, int)
RETURNS int
AS
$$
  BEGIN
       RETURN 
       	CASE WHEN $1 = $2 THEN $1 ELSE $2 END;
  END;
$$ LANGUAGE 'plpgsql' IMMUTABLE;

Check the same query again:
You can find ‘Index Scan’ which is expected and correct behaviour.
EXPLAIN

SELECT *FROM tbl_ItemTransactions
WHERE TranID = fn_getvalues (50,80);

'Index Scan using idx_tranid on tbl_itemtransactions  (cost=0.42..8.44 rows=1 width=18)'
'  Index Cond: (tranid = 80)'

In this post, I am sharing a script for getting the list of objects of a particular User (Owner of the Object) in PostgreSQL.
Generally, PostgreSQL DBAs are using this kind of script for auditing purpose and object tracking purpose.

Using the below script, you can get the list object for a particular user:

I used a ‘postgres’ default user in WHERE condition, which you can edit as per your user.

select 
    nsp.nspname as SchemaName
    ,cls.relname as ObjectName 
    ,rol.rolname as ObjectOwner
    ,case cls.relkind
        when 'r' then 'TABLE'
        when 'm' then 'MATERIALIZED_VIEW'
        when 'i' then 'INDEX'
        when 'S' then 'SEQUENCE'
        when 'v' then 'VIEW'
        when 'c' then 'TYPE'
        else cls.relkind::text
    end as ObjectType
from pg_class cls
join pg_roles rol 
	on rol.oid = cls.relowner
join pg_namespace nsp 
	on nsp.oid = cls.relnamespace
where nsp.nspname not in ('information_schema', 'pg_catalog')
    and nsp.nspname not like 'pg_toast%'
    and rol.rolname = 'postgres'  
order by nsp.nspname, cls.relname