I recently started work with Greenplum Distributed RDBMS. Internally, it is using PostgreSQL 8.2 which is an ancient version of PostgreSQL.

I shared a couple of articles on finding the fragmentation in PostgreSQL, but none of my scripts worked for Greenplum PostgreSQL 8.2.

I have Posted one question on StackOverflow. Please check this link.

Finally, I found a script for finding the fragmentation of Greenplum tables which is very important for Greenplum DBA because Greenplum is a based on MVCC architecture which generates dead rows at every UPDATE / DELETE.

What is Multi Version Concurrency Control (MVCC)

You can use below script for finding the wasted space of the tables and can execute VACUUM ANALYZE on those tables.

select
	current_database() as dbname
	, relid
	, schemaname
	, tablename
	, round(bloat,1) as bloat_ratio
	, pg_size_pretty(expbytes) as expected_size
	, pg_size_pretty(relbytes) as relation_size
	, pg_size_pretty(wastedbytes) as wasted_space
	, round(1.0 * relbytes / nullif((sum(relbytes) over ())::bigint,0),4) as relative_size
	, round((bloat-1) * (1.0 * relbytes / nullif((sum(relbytes) over ())::bigint, 0)), 8) as concern
from
(
	SELECT
		relid
		, schemaname
		, tablename
		, CASE WHEN otta=0 THEN 
			0.0 
			ELSE 
			sml.relpages/otta::numeric 
			END AS bloat
		, reltuples::bigint
		, relpages::bigint
		, otta
		, (bs*otta)::bigint as expbytes
		, CASE WHEN relpages < otta THEN 0 ELSE 
		(bs*(sml.relpages-otta))::bigint 
		END AS wastedbytes
		, CASE WHEN relpages < otta THEN 0 ELSE relpages::bigint - otta END AS wastedpages
		, (bs*relpages)::bigint as relbytes
	FROM
	(
		SELECT
			schemaname
			, tablename
			, cc.oid as relid
			, cc.reltuples
			, cc.relpages
			, bs
			, CEIL((cc.reltuples*((datahdr+ma-(CASE WHEN datahdr%ma=0 THEN ma ELSE datahdr%ma END))+nullhdr2+4))/(bs-20::float)) AS otta 
		FROM 
		(
			SELECT
				ma
				, bs
				, schemaname
				, tablename
				, (datawidth+(hdr+ma-(case when hdr%ma=0 THEN ma ELSE hdr%ma END)))::numeric AS datahdr
				, (maxfracsum*(nullhdr+ma-(case when nullhdr%ma=0 THEN ma ELSE nullhdr%ma END))) AS nullhdr2
			FROM
			(
				SELECT
					schemaname
					, tablename
					, hdr
					, ma
					, bs
					, SUM((1-s.null_frac)*avg_width) AS datawidth
					, MAX(s.null_frac) AS maxfracsum
					, hdr+(1+(count(case when s.null_frac<>0 then 1 end))/8) as nullhdr
				FROM 
				pg_stats s
				cross join 
				(
					SELECT
					current_setting('block_size')::numeric AS bs
					, CASE WHEN substring(version(),12,3) IN ('8.0','8.1','8.2') THEN 27 ELSE 23 END AS hdr
					, CASE WHEN version() ~ 'mingw32' THEN 8 ELSE 4 END AS ma
				) AS constants
				GROUP BY schemaname, tablename, hdr, ma, bs
			) AS foo
		) AS rs
		JOIN pg_class cc 
		ON
		cc.relname = rs.tablename 
		and cc.relkind = 'r'
		JOIN pg_namespace nn 
		ON
		cc.relnamespace = nn.oid 
		AND nn.nspname = rs.schemaname 
	) AS sml
) wrapper
where wastedbytes > 2*1024*1024 and bloat >= 1.4order by (bloat-1) * (1.0 * relbytes / nullif((sum(relbytes) over ())::bigint, 0)) desc;

Taken reference from this article

In this post, I am sharing an important gp_toolkit view to finding a list of a table on which VACUUM is required to be run in Greenplum Database Server.

Greenplum is based on PostgreSQL 8.2 and MVCC architecture. It generates dead tuples at every UPDATE and DELETE which is a fragmentation in the form of dead tuples which we can remove by executing a VACUUM/VACUUM FULL command.

What is Multi Version Concurrency Control (MVCC)

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_bloat_diag is one of the views of a gp_toolkit schema which gives us a list of tables which may require a VACUUM to be run.

It returns the actual number of pages and the expected number of pages. If a count of an actual number of pages is greater than expected number of pages, that table requires VACUUM.

If you want to re-use the wasted space of table, execute VACUUM FULL on it which requires ACCESS EXCLUSIVE LOCK on a table because it re-writes all the data pages from the scratch.

If you execute only VACUUM, you cannot reclaims the free space of table.
You must execute ANALYZE after the execution of VACUUM/VACUUM FULL for updating database statistics.

Below is a query:

SELECT * FROM gp_toolkit.gp_bloat_diag;

After executing the above query, you can get the list of six columns. The important columns are “bdirelpages” and “bdiexppages” means real pages and expected pages.

If you find your table in the result set, your table requires VACUUM/VACUUM FULL.

In this post, I am sharing an important gp_toolkit view to finding a list of a table on which ANALYZE is required to be run in Greenplum Database Server.

An ANALYZE command is used to update the object or database statistics information. The query planner always uses the statistics for building the execution plans.

A gp_toolkit is an administrative schema of Greenplum Database server. It provides the list of various views to monitor the Greenplum server.

A gp_stats_missing is one of the views of a gp_toolkit schema which gives us a list of tables which may require an ANALYZE to be run.

Below is a query:

SELECT * FROM gp_toolkit.gp_stats_missing;

After executing the above query, you can get the list of five columns. A column “smisize” is an important column to check the status of table statistics.

If it contains value ‘f’, a table does not have row count and row sizing statistics recorded in the system catalog, which indicates that the table needs to be analyzed.

The meaning of other columns like “simcols”: number of columns in the table, “simrecs”: number of rows in the table.

Greenplum is based on MPP (massively parallel processing) architecture. It built on shared nothing mode parallel processing in which all child segments own separate resources.

Database Theory: What is Parallel Query Processing (Parallel Database System)?

When you are taking database backup in Greenplum, it executes a backup process in different stages and executes in parallel.

For GP DBA, this is very important for knowing that in which machine or segment backup process is running. They may have few other questions like Is backup process completed for a master segment?, If it is completed for a master segment, in which child segment it is running currently.

You can access this article for troubleshooting Greenplum running database backup process.

You can start Greenplum full database backup using below command:

gpcrondump -x database_name -u /home/gpadmin/dbrnd_backup/

It first takes a backup of the master node (all metadata or pg_catalog), while during this time you cannot perform any DDL or any other operation on Greenplum database server like CREATE TABLE, DROP TABLE, TRUNCATE TABLE.
It takes only a few minutes to complete the backup of a master node.

How can you check that, Is Greenplum database backup process running on master node?

The first step is to check if the message “Starting Dump process” is on the screen
After this message, you can find a subsequent message “Releasing pg_class lock” that means backup process completed for master segment (meta data backup).

You can measure the time difference between “Starting Dump process” and “Releasing pg_class lock” that is the total time taken for master segment backup.

If you are unable to find “Releasing pg_class lock” on the screen and find only “Starting Dump process” for a long time, there is 99% problem with a master segment and problem like database lock, network issue, disk failure.

How can we check that, Is Greenplum database backup process running on child nodes?

After taken a master node back up, a gp_dump command will be sent to all the segments to start a copy of the data using COPY command.

We can check the running processes in child nodes and can find COPY command and db_dumps directory in the process list. You can get the list of Linux running processes using a command: ps -ef.
Below is a sample process message:

postgres: port 40002, gpadmin database_name [local] con808 [local] cmd880 COPY

Important Note:

Please do not kill long running Greenplum backup from only master node because it may continue the backup process on child segments. Your one wrong kill command can fail your whole Greenplum cluster.

If you guys know how to kill long running backup in Greenplum, please share full detail with me because I am also looking for the solution for killing backup process at all child segments.

In this post, I am sharing a script to find a list of blocked and deadlock queries by other blocking queries of Greenplum Database Server.

You can visit below article to know more about, what is a lock, deadlock, block and timeout?

Database Theory: What is Timeout, Lock, Block, and Deadlock in a Database System?

If two queries are trying to get the same resource, both are depended on each other and generates block state first and then deadlock.

In the below script, you can find a list of blocked queries and blocker queries with a value of time age.
You can find a process_id of blocker query which we can kill for avoiding the deadlock situation.

SELECT
	kl.pid as blocking_pid
	,ka.usename as blocking_user
	,ka.current_query as blocking_query
	,bl.pid as blocked_pid
	,a.usename as blocked_user
	,a.current_query as blocked_query
	,to_char(age(now(), a.query_start),'HH24h:MIm:SSs') as age
FROM pg_catalog.pg_locks bl
JOIN pg_catalog.pg_stat_activity a
	ON bl.pid = a.procpid
JOIN pg_catalog.pg_locks kl
	ON bl.locktype = kl.locktype
	and bl.database is not distinct from kl.database
	and bl.relation is not distinct from kl.relation
	and bl.page is not distinct from kl.page
	and bl.tuple is not distinct from kl.tuple
	and bl.transactionid is not distinct from kl.transactionid
	and bl.classid is not distinct from kl.classid
	and bl.objid is not distinct from kl.objid
	and bl.objsubid is not distinct from kl.objsubid
	and bl.pid <> kl.pid
JOIN pg_catalog.pg_stat_activity ka
	ON kl.pid = ka.procpid
WHERE kl.granted and not bl.granted
ORDER BY a.query_start;

In this post, I am sharing a script to find long running queries or sessions with the lock information of Greenplum database server.

In the below script, you can find the age column which gives you total execution time of running sessions. You can also find query lock related details.

select 
	age(now(),pg_stat_activity.query_start) as "age",
	pg_stat_activity.datname, pg_stat_activity.usename,
	pg_stat_activity.client_addr, pg_stat_activity.application_name,
	pg_locks.transactionid, pg_locks.transaction,
	pg_stat_activity.procpid, pg_stat_activity.query_start,
	pg_locks.mode, pg_stat_activity.current_query
from  pg_stat_activity, pg_locks
where pg_locks.pid = pg_stat_activity.procpid
and procpid <> pg_backend_pid()
and pg_locks.transactionid = pg_locks.transaction
order by 1 desc;

In this post, I am sharing a script to check the list of running queries or statements which are waiting in the Resource Queues of Greenplum.

The Workload management of Greenplum means – Set priorities of database user’s queries using Resource Queues. If you create RQ with the limit of 10 concurrent queries and user is trying to execute more than 10 queries at a time then few RQ keeps few queries in the waiting mode.

Using below query, you can get the list of running queries which are in the waiting mode.

SELECT
	rolname
	,rsqname
	,pid
	,granted
	,current_query
	,datname
FROM pg_roles, gp_toolkit.gp_resqueue_status
,pg_locks, pg_stat_activity
WHERE pg_roles.rolresqueue=pg_locks.objid
	AND pg_locks.objid=gp_toolkit.gp_resqueue_status.queueid
	AND pg_stat_activity.procpid=pg_locks.pid
	AND pg_stat_activity.usename=pg_roles.rolnam

In this post, I am sharing few utility commands which we can use to start and stop the Greenplum Database Cluster.

Start the database cluster:
The gpstart utility starts the database on Greenplum cluster.

gpstart

Restart the database cluster:

gpstop -r

Reload the configuation:
It reloads the pg_hba.conf and postgresql.conf file.

gpstop -u

Stop the database cluster:
It stops all the processes, master-child segment instances, parallel worker.

gpstop

Stop the database cluster in fast mode:

gpstop -M fast

In this post, I am sharing a script to find the average running time and average resource utilization of running queries in Greenplum.

I am using the table of GPCC which are available in GPPERFMON database. So if you don’t configure GPCC, you can’t use below query.

Using below query, you can find the total average run time, average CPU, average Memory occupied by long-running queries which took more than 20 minutes and checking for only last 10 days.
As per your requirement, you can change the minutes and days filter value.

select
	sh.ctime
	,query_text
	,username
	,db as DatabaseName
	,rsqname as ResourceQueueName
	,avg(tfinish-tstart) as AverageRunTime
	,count(*) as TotalExecution
	,round(avg(100 - cpu_idle)::numeric,2) as AverageCPUUsed
	,round(max(100 - cpu_idle)::numeric,2) as MaxCPUUsed
	,round(avg(mem_actual_used)/power(1024,3)::numeric,2) as AverageMemoryUsed
	,round(max(mem_actual_used)/power(1024,3)::numeric,2) as MaxMemoryUsed
from system_history sh, queries_history qh
where sh.ctime between date_trunc('day',localtimestamp - interval '10 days')
	and date_trunc('day',localtimestamp)
	and sh.ctime=qh.ctime
	and db not in ('gpperfmon')
	and date_part('hour',tfinish - tstart)*60 + date_part('minute',tfinish - tstart) > 20
group by sh.ctime,query_text,username, db,rsqname;

In this post, I am sharing a few scripts for checking the size of Database, Schema and Table of Greenplum Server.

Here, I am doing this exercise using the views of the gp_toolkit schema of Greenplum.

Check the size of database in MB, GB and TB:

SELECT sodddatname, (sodddatsize/1048576) AS Sizein_MB
FROM gp_toolkit.gp_size_of_database;
 
SELECT sodddatname, (sodddatsize/1073741824) AS Sizein_GB
FROM gp_toolkit.gp_size_of_database;
 
SELECT sodddatname, (sodddatsize/1073741824)/1024 AS Sizein_TB
FROM gp_toolkit.gp_size_of_database;

Checkt the size of database in MB, GB and TB:

SELECT sosdnsp, (sosdschematablesize/1048576) AS Sizein_MB
FROM gp_toolkit.gp_size_of_schema_disk;
 
SELECT sosdnsp, (sosdschematablesize/1073741824) AS Sizein_GB
FROM gp_toolkit.gp_size_of_schema_disk;
 
SELECT sosdnsp, (sosdschematablesize/1073741824)/1024 AS Sizein_TB
FROM gp_toolkit.gp_size_of_schema_disk;

Check the size of any Table or Index:

SELECT pg_size_pretty(pg_total_relation_size('schema_name.table_name'));