In this post, I am sharing an important gp_toolkit view to finding a list of a table on which VACUUM is required to be run in Greenplum Database Server.

Greenplum is based on PostgreSQL 8.2 and MVCC architecture. It generates dead tuples at every UPDATE and DELETE which is a fragmentation in the form of dead tuples which we can remove by executing a VACUUM/VACUUM FULL command.

What is Multi Version Concurrency Control (MVCC)

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_bloat_diag is one of the views of a gp_toolkit schema which gives us a list of tables which may require a VACUUM to be run.

It returns the actual number of pages and the expected number of pages. If a count of an actual number of pages is greater than expected number of pages, that table requires VACUUM.

If you want to re-use the wasted space of table, execute VACUUM FULL on it which requires ACCESS EXCLUSIVE LOCK on a table because it re-writes all the data pages from the scratch.

If you execute only VACUUM, you cannot reclaims the free space of table.
You must execute ANALYZE after the execution of VACUUM/VACUUM FULL for updating database statistics.

Below is a query:

SELECT * FROM gp_toolkit.gp_bloat_diag;

After executing the above query, you can get the list of six columns. The important columns are “bdirelpages” and “bdiexppages” means real pages and expected pages.

If you find your table in the result set, your table requires VACUUM/VACUUM FULL.

In this post, I am sharing an important gp_toolkit view to finding a list of a table on which ANALYZE is required to be run in Greenplum Database Server.

An ANALYZE command is used to update the object or database statistics information. The query planner always uses the statistics for building the execution plans.

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_stats_missing is one of the views of a gp_toolkit schema which gives us a list of tables which may require an ANALYZE to be run.

Below is a query:

SELECT * FROM gp_toolkit.gp_stats_missing;

After executing the above query, you can get the list of five columns. A column “smisize” is an important column to check the status of table statistics.

If it contains value ‘f’, a table does not have row count and row sizing statistics recorded in the system catalog, which indicates that the table needs to be analyzed.

The meaning of other columns like “simcols”: number of columns in the table, “simrecs”: number of rows in the table.

In this post, I am sharing a script to check the list of running queries or statements which are waiting in the Resource Queues of Greenplum.

The Workload management of Greenplum means – Set priorities of database user’s queries using Resource Queues. If you create RQ with the limit of 10 concurrent queries and user is trying to execute more than 10 queries at a time then few RQ keeps few queries in the waiting mode.

Using below query, you can get the list of running queries which are in the waiting mode.

SELECT
	rolname
	,rsqname
	,pid
	,granted
	,current_query
	,datname
FROM pg_roles, gp_toolkit.gp_resqueue_status
,pg_locks, pg_stat_activity
WHERE pg_roles.rolresqueue=pg_locks.objid
	AND pg_locks.objid=gp_toolkit.gp_resqueue_status.queueid
	AND pg_stat_activity.procpid=pg_locks.pid
	AND pg_stat_activity.usename=pg_roles.rolnam

In this post, I am sharing a few scripts for checking the size of Database, Schema and Table of Greenplum Server.

Here, I am doing this exercise using the views of the gp_toolkit schema of Greenplum.

Check the size of database in MB, GB and TB:

SELECT sodddatname, (sodddatsize/1048576) AS Sizein_MB
FROM gp_toolkit.gp_size_of_database;
 
SELECT sodddatname, (sodddatsize/1073741824) AS Sizein_GB
FROM gp_toolkit.gp_size_of_database;
 
SELECT sodddatname, (sodddatsize/1073741824)/1024 AS Sizein_TB
FROM gp_toolkit.gp_size_of_database;

Checkt the size of database in MB, GB and TB:

SELECT sosdnsp, (sosdschematablesize/1048576) AS Sizein_MB
FROM gp_toolkit.gp_size_of_schema_disk;
 
SELECT sosdnsp, (sosdschematablesize/1073741824) AS Sizein_GB
FROM gp_toolkit.gp_size_of_schema_disk;
 
SELECT sosdnsp, (sosdschematablesize/1073741824)/1024 AS Sizein_TB
FROM gp_toolkit.gp_size_of_schema_disk;

Check the size of any Table or Index:

SELECT pg_size_pretty(pg_total_relation_size('schema_name.table_name'));

In this post, I am sharing two different scripts to check the full size of Greenplum DCA.
To monitor the disk space is a ubiquitous task of DBA. DBA can use below script and can automate the report on disk size notification/alert.

There are different options to check the disk size of the server; below I prepared two queries using gpperfmon database and gp_toolkit schema.

Use gpperfmon Database:

select
	hostname
	,filesystem
	,round(total_bytes/power(1024,4)::numeric,2) as TotalSize_inTB
	,round(bytes_used/power(1024,4)::numeric,2) as UsedSize_inTB
	,round(bytes_available/power(1024,4)::numeric,2) as AvailableSize_inTB
from diskspace_now
where filesystem in ('/data1','/data2')
ORDER BY 4 DESC

Use gp_toolkit Schema:

select
	distinct dfhostname
	,dfdevice
	,dfspace
from gp_toolkit.gp_disk_free

In this post, I am sharing a script to find the total usage of work spill files for each running query in Greenplum.

Whenever we are doing hash aggregation operation like GROUP BY or DISTINCT on billions of data, query executor holds this data in temporary files which called work spill files.

We should optimize our query to reduce the maximum use of work spill files because it degrades the overall performance and also requires more disk space.

Using below script, we can monitor the usage of work spill file for each running query and we can also optimize the long running queries.

SELECT
	gwupq.datname
	,psa.usename
	,gwupq.procpid
	,gwupq.sess_id
	,sum(gwupq.size)/1024::float as total_spill_size_kb
	,sum(gwupq.numfiles) as total_spill_file
FROM  gp_toolkit.gp_workfile_usage_per_query as gwupq
inner join pg_stat_activity psa
	on gwupq.procpid = psa.procpid
	and gwupq.sess_id = psa.sess_id
GROUP BY 1,2,3,4;

Greenplum is a based on MVCC architecture that means user’s action like UPDATE will insert a new record in a database.
Greenplum Data warehouse can manage billions of data, but it doesn’t mean that user can generate or UPDATE random data set.
Periodically, The Greenplum DBA should check the size of database schemas and their appropriate usage.

In this post, I shared different scripts to check the size of Greenplum Database Schemas.

Option 1:

select *
from gp_toolkit.gp_size_of_schema_disk;

Option 2:

select
	schemaname
	,round(sum(pg_total_relation_size(schemaname||'.'||tablename))) SchemaSize
from pg_tables group by 1;

Option 3:

SELECT
	schema_name
	,pg_size_pretty(sum(table_size)::bigint)
FROM
(
	SELECT pg_catalog.pg_namespace.nspname as schema_name,
	pg_relation_size(pg_catalog.pg_class.oid) as table_size
	FROM pg_catalog.pg_class
	JOIN pg_catalog.pg_namespace ON relnamespace = pg_catalog.pg_namespace.oid
) t
GROUP BY schema_name
ORDER BY schema_name;

In this post, I am sharing a script to find the total usage of work spill files for all the segments of Greenplum.

Whenever we are doing hash aggregation operations like GROUP BY or DISTINCT on billions of data, query executor holds this data in temporary files which called work spill files.

We should optimize our query to reduce the maximum use of work spill files because it degrades the overall performance and also requires more disk space.
Using below query, you can check the total number of spill files and their size for each segment.

SELECT
                gwe.datname as DatabaseName
                ,psa.usename as UserName
                ,gwe.procpid as ProcessID
                ,gwe.sess_id as SessionID
                ,sc.hostname as HostName
                ,sum(size)/1024::float as SizePerHost
                ,sum(numfiles) NumOfFilesPerHost
FROM  gp_toolkit.gp_workfile_entries as gwe
inner join pg_stat_activity as psa
                on psa.procpid = gwe.procpid
                and psa.sess_id = gwe.sess_id,
gp_segment_configuration as sc
,pg_filespace_entry as fe
,pg_database as d
WHERE fe.fsedbid=sc.dbid
                AND gwe.segid=sc.content
                AND gwe.datname=d.datname
                AND sc.role='p'
group by
                gwe.datname
                ,psa.usename
                ,gwe.procpid
                ,gwe.sess_id
                ,sc.hostname
ORDER BY
                gwe.datname
                ,psa.usename
                ,gwe.procpid
                ,gwe.sess_id
                ,sc.hostname