In this post, I am sharing a script for checking the status of AutoVacuum for all the PostgreSQL Tables.

If you don’t know about the MVCC architecture, you must visit the below article. Because of MVCC, we have to work on dead tuples which are generated by transactions in PostgreSQL.

What is Multi Version Concurrency Control (MVCC)

You can configure auto-vacuum on tables, but periodically you should check the status of it. Use below script for checking the status of Autovacuum.

PostgreSQL: Short note on VACUUM, VACUUM FULL and ANALYZE

PostgreSQL: Force Autovacuum for running Aggressive

SELECT 
    schemaname
    ,relname
    ,n_live_tup
    ,n_dead_tup
    ,last_autovacuum
FROM pg_stat_all_tables
ORDER BY n_dead_tup
    /(n_live_tup
      * current_setting('autovacuum_vacuum_scale_factor')::float8
      + current_setting('autovacuum_vacuum_threshold')::float8)
     DESC

In this post, I am sharing an important gp_toolkit view to finding a list of a table on which VACUUM is required to be run in Greenplum Database Server.

Greenplum is based on PostgreSQL 8.2 and MVCC architecture. It generates dead tuples at every UPDATE and DELETE which is a fragmentation in the form of dead tuples which we can remove by executing a VACUUM/VACUUM FULL command.

What is Multi Version Concurrency Control (MVCC)

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_bloat_diag is one of the views of a gp_toolkit schema which gives us a list of tables which may require a VACUUM to be run.

It returns the actual number of pages and the expected number of pages. If a count of an actual number of pages is greater than expected number of pages, that table requires VACUUM.

If you want to re-use the wasted space of table, execute VACUUM FULL on it which requires ACCESS EXCLUSIVE LOCK on a table because it re-writes all the data pages from the scratch.

If you execute only VACUUM, you cannot reclaims the free space of table.
You must execute ANALYZE after the execution of VACUUM/VACUUM FULL for updating database statistics.

Below is a query:

SELECT * FROM gp_toolkit.gp_bloat_diag;

After executing the above query, you can get the list of six columns. The important columns are “bdirelpages” and “bdiexppages” means real pages and expected pages.

If you find your table in the result set, your table requires VACUUM/VACUUM FULL.

This is one of the very important post for all PostgreSQL Database Professionals.

In this post, I am sharing a small, but very powerful script to know about the Live Tuples (Rows) and Dead Tuples (Rows) of the PostgreSQL Object like: Tables and Indexes.

If you don’t know about the MVCC (Multi Version Concurrency Control), Please visit this article.

What is Multi Version Concurrency Control (MVCC)

PostgreSQL is based on MVCC Architecture.

In MVCC Architecture, When you update or delete any row, Internally It creates the new row and mark old row as unused.

This kind of data, we call as Dead Tuples or Dead Rows. (We can also say like, This is an internal fragmentation).

Periodically, We should find dead rows of the object and we should remove it using VACUUM techniques of PostgreSQL.
By this way, we can increase the overall performance of PostgreSQL Database Server.

PostgreSQL: Short note on VACUUM, VACUUM FULL and ANALYZE

Find out Live Tuples or Dead Tuples using two different scripts.

Using system function and pg_class:

SELECT 
	relname AS ObjectName
	,pg_stat_get_live_tuples(c.oid) AS LiveTuples
	,pg_stat_get_dead_tuples(c.oid) AS DeadTuples
FROM pg_class c;

Using pg_stat_user_tables:

SELECT 
	relname AS TableName
	,n_live_tup AS LiveTuples
	,n_dead_tup AS DeadTuples
FROM pg_stat_user_tables;

How to perform VACUUM FULL without disk space?

The above question is common for all PostgreSQL DBA. This is really a very important topic that I am going to discuss in this post.

We require VACUUM and VACUUM FULL in PostgreSQL because of MVCC Architecture.
You can visit this article for MVCC:

Postgres VACUUM FULL reclaims all free space released by VACUUM by removing dead rows physically.

This method also requires extra disk space, since it writes a new copy of the table and doesn’t release the old copy until the operation is complete.

In one of our Postgres Database Server, we require VACUUM FULL on one table, and this table size is 13GB, and we don’t have enough free space on hard-disk.

I worked around this problem and created one of the best solutions to resolve this problem.

To solve this problem, we need additional storage like any other network drive or portable hard disk.
Please do not forget that VACUUM FULL requires an exclusive lock on the table so during this operation your table cannot be accessible.

Now, add new hard disk and make it as table-space.

Create a new tablespace:

CREATE TABLESPACE temptablespace LOCATION '/path/../';

Check your table current tablespace:

SELECT tablespace FROM pg_tables WHERE tablename = 'mybigtable';

If it is NULL, it has a default tablespace.

Move table to new tablespace:

ALTER TABLE mybigtable SET TABLESPACE temptablespace;

Perform VACUUM FULL:

VACUUM FULL mybigtable;

Move table to old tablespace:(moving to pg_default)

ALTER TABLE mybigtable SET TABLESPACE pg_default;

Drop that temp table space:

DROP TABLESPACE temptablespace;

VACUUM, VACUUM FULL and ANALYZE: These are the maintenance related commands of PostgreSQL which requires frequent execution because PostgreSQL based on MVCC architecture where every UPDATE and DELETE generates dead rows or dead tuples as an internal fragmentation.

If you don’t about the MVCC, you must visit the below article.
What is MVCC?

VACUUM:

VACUUM reclaims storage occupied by dead tuples.
In the MCVV architecture tuples that deleted which is not physically remove. It presents in a disk until a VACUUM is done.

If you are executing frequent UPDATE/DELETE on your table, you must find the fragmentation and must execute VACUUM on it.
VACUUM does not require an exclusive lock on the table.
Syntax:

VACUUM Table_Name;

VACUUM FULL:

Whatever space reclaims by VACUUM that space might require to use for next storage.
VACUUM FULL rewrites the entire table with data and releases all fragmented space of an old table.

We should avoid the VACUUM FULL because it shrinks the whole table and writes everything into the new disk block which requires more resources and disk space to complete this operation.

If you find a huge number of dead rows, you should execute VACUUM FULL on that table.
VACUUM FULL requires an exclusive lock on the table and also require a free disk space as same as your table size.
Syntax:

VACUUM FULL Table_Name;

ANALYZE:

After execution of VACUUM or VACUUM FULL, you must execute this command to update the statistics.
ANALYZE updates all require statistics and stores the results in the pg_statistic system catalog.

The Query Planner uses this database statistics information to prepare an efficient query execution plan.
You can also execute VACUUM ANALYZE, which executes the first VACUUM and then executes ANALYZE.
Syntax:

ANALYZE Table_Name;
VACUUM ANALYZE Table_Name;