In this post, I am sharing a script for checking the status of AutoVacuum for all the PostgreSQL Tables.

If you don’t know about the MVCC architecture, you must visit the below article. Because of MVCC, we have to work on dead tuples which are generated by transactions in PostgreSQL.

What is Multi Version Concurrency Control (MVCC)

You can configure auto-vacuum on tables, but periodically you should check the status of it. Use below script for checking the status of Autovacuum.

PostgreSQL: Short note on VACUUM, VACUUM FULL and ANALYZE

PostgreSQL: Force Autovacuum for running Aggressive

SELECT 
    schemaname
    ,relname
    ,n_live_tup
    ,n_dead_tup
    ,last_autovacuum
FROM pg_stat_all_tables
ORDER BY n_dead_tup
    /(n_live_tup
      * current_setting('autovacuum_vacuum_scale_factor')::float8
      + current_setting('autovacuum_vacuum_threshold')::float8)
     DESC

In this post, I am sharing an important gp_toolkit view to finding a list of a table on which VACUUM is required to be run in Greenplum Database Server.

Greenplum is based on PostgreSQL 8.2 and MVCC architecture. It generates dead tuples at every UPDATE and DELETE which is a fragmentation in the form of dead tuples which we can remove by executing a VACUUM/VACUUM FULL command.

What is Multi Version Concurrency Control (MVCC)

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_bloat_diag is one of the views of a gp_toolkit schema which gives us a list of tables which may require a VACUUM to be run.

It returns the actual number of pages and the expected number of pages. If a count of an actual number of pages is greater than expected number of pages, that table requires VACUUM.

If you want to re-use the wasted space of table, execute VACUUM FULL on it which requires ACCESS EXCLUSIVE LOCK on a table because it re-writes all the data pages from the scratch.

If you execute only VACUUM, you cannot reclaims the free space of table.
You must execute ANALYZE after the execution of VACUUM/VACUUM FULL for updating database statistics.

Below is a query:

SELECT * FROM gp_toolkit.gp_bloat_diag;

After executing the above query, you can get the list of six columns. The important columns are “bdirelpages” and “bdiexppages” means real pages and expected pages.

If you find your table in the result set, your table requires VACUUM/VACUUM FULL.

In this post, I am sharing an important gp_toolkit view to finding a list of a table on which ANALYZE is required to be run in Greenplum Database Server.

An ANALYZE command is used to update the object or database statistics information. The query planner always uses the statistics for building the execution plans.

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_stats_missing is one of the views of a gp_toolkit schema which gives us a list of tables which may require an ANALYZE to be run.

Below is a query:

SELECT * FROM gp_toolkit.gp_stats_missing;

After executing the above query, you can get the list of five columns. A column “smisize” is an important column to check the status of table statistics.

If it contains value ‘f’, a table does not have row count and row sizing statistics recorded in the system catalog, which indicates that the table needs to be analyzed.

The meaning of other columns like “simcols”: number of columns in the table, “simrecs”: number of rows in the table.

In this post, I am sharing a script to find a list of tables with the information like the last action_date of VACUUM and ANALYZE in Greenplum.

Before executing VACUUM or ANALYZE on tables, we should check the last execution time of it. If it recently executed, we should skip those tables for VACUUM or ANALYZE.
In below script, I am finding this information from pg_stat_last_operation.

select
       pn.nspname
       ,pc.relname
       ,pslo.staactionname
       ,pslo.stasubtype
       ,pslo.statime as action_date
from pg_stat_last_operation pslo
right outer join pg_class pc
                on pc.oid = pslo.objid and pslo.staactionname in ('VACUUM','ANALYZE')
join pg_namespace pn
                on pn.oid = pc.relnamespace
where pc.relkind IN ('r','s')
AND pc.relstorage IN ('h', 'a', 'c');

In any RDBMS system, Different type of Statistics is very important for Query Optimizer and Planner to choose and generate the best Query Execution Plan.

PostgreSQL Database Server also managing different types of Statistic which are used by the Query Optimizer to generate the best Query Execution Plan.

Statistical data should be updated, Why?
For Example,

We have one table with 100000 rows, but table statistic data are not updated and showing only 1000 tuples information.

Query optimizer always looks into Statistic tables, It never looks into direct table files.

So in this situation, It generates the wrong Query Execution Plan because It has only 1000 rows information, but actually there are 100000 rows.

PostgreSQL Database Server is mainly using two Statistical Table.

SELECT *FROM pg_class;
SELECT *FROM pg_stats;

You can also query these tables and can find information about Tables, Indexes, Pages and others.

When you look into table pg_class, you can find “reltuples” column. You can compare this column value with your actual table record count.

If you find any mismatch, for that table statistic information is not updated.

Execute Analyze command:

When you find that Statistical information is not updated, you can execute the ANALYZE command on the Database or Table.
After executing the ANALYZE command, It updates all require statistics.

PostgreSQL: Short note on VACUUM, VACUUM FULL and ANALYZE

VACUUM, VACUUM FULL and ANALYZE: These are the maintenance related commands of PostgreSQL which requires frequent execution because PostgreSQL based on MVCC architecture where every UPDATE and DELETE generates dead rows or dead tuples as an internal fragmentation.

If you don’t about the MVCC, you must visit the below article.
What is MVCC?

VACUUM:

VACUUM reclaims storage occupied by dead tuples.
In the MCVV architecture tuples that deleted which is not physically remove. It presents in a disk until a VACUUM is done.

If you are executing frequent UPDATE/DELETE on your table, you must find the fragmentation and must execute VACUUM on it.
VACUUM does not require an exclusive lock on the table.
Syntax:

VACUUM Table_Name;

VACUUM FULL:

Whatever space reclaims by VACUUM that space might require to use for next storage.
VACUUM FULL rewrites the entire table with data and releases all fragmented space of an old table.

We should avoid the VACUUM FULL because it shrinks the whole table and writes everything into the new disk block which requires more resources and disk space to complete this operation.

If you find a huge number of dead rows, you should execute VACUUM FULL on that table.
VACUUM FULL requires an exclusive lock on the table and also require a free disk space as same as your table size.
Syntax:

VACUUM FULL Table_Name;

ANALYZE:

After execution of VACUUM or VACUUM FULL, you must execute this command to update the statistics.
ANALYZE updates all require statistics and stores the results in the pg_statistic system catalog.

The Query Planner uses this database statistics information to prepare an efficient query execution plan.
You can also execute VACUUM ANALYZE, which executes the first VACUUM and then executes ANALYZE.
Syntax:

ANALYZE Table_Name;
VACUUM ANALYZE Table_Name;