If you don’t know about that data pages, you must visit below article.
What is MVCC ?

Because of MVCC architecture, PostgreSQL generates dead rows at every UPDATE and DELETE.

You must remove the fragmentation by executing VACUUM ANALYZE command which removes the dead rows and update the related database statistics.

You can also use VACUUM VERBOSE to troubleshoot total number of data pages, total number of removable dead row version while executing a VACUUM ANALYZE.
(Check your query message window, where you can find detailed information generated by VACUUM VERBOSE.)

This is really very helpful information for PostgreSQL DBA to make sure about the fragmentation.

I have prepared a detailed demonstration of this.

Create a table with sample data:

CREATE TABLE tbl_ItemTransactions 
 (
     TranID SERIAL
     ,TransactionDate TIMESTAMPTZ
     ,TransactionName TEXT
 ) DISTRIBUTED BY (TranID);

INSERT INTO tbl_ItemTransactions 
(TransactionDate, TransactionName)
SELECT x, 'ggkproduct' 
FROM generate_series('2015-01-01 00:00:00'::timestamptz, '2016-08-01 00:00:00'::timestamptz,'5 seconds'::interval) a(x);

Check the size of table:

SELECT pg_size_pretty(pg_total_relation_size('tbl_ItemTransactions')) AS TableSize;

The size is: "496 MB"

Execute VACUUM VERBOSE to get data pages and dead row versions related information:

VACUUM VERBOSE ANALYZE tbl_ItemTransactions;

"tbl_itemtransactions": found 0 removable, 2496960 nonremovable row versions in 3970 pages  (seg0 sdw1:40000 pid=17647)

Execute one DELETE to generate few dead tuples:

DELETE FROM tbl_ItemTransactions 
WHERE TransactionDate BETWEEN '2016-04-01' AND '2016-08-01';

Now, examine the result of VACUUM VERBOSE where you can find 527040 removable rows:
You can find total 3970 pages and 527040 removable rows.

VACUUM VERBOSE ANALYZE tbl_ItemTransactions;

INFO:  "tbl_itemtransactions": found 527040 removable, 1969920 nonremovable row versions in 3970 pages  (seg1 sdw1:40001 pid=17649)
839 pages contain useful free space.

Now, execute one more time and you can find 3132 pages and 0 removable rows:

VACUUM VERBOSE ANALYZE tbl_ItemTransactions;

"tbl_itemtransactions": found 0 removable, 1969920 nonremovable row versions in 3132 pages  (seg0 sdw1:40000 pid=17647)
1 pages contain useful free space.

Check the size of table:

SELECT pg_size_pretty(pg_total_relation_size('tbl_ItemTransactions')) AS TableSize;

The size is: "392 MB"

Before reading this article, I would suggest please read about MVCC because of PostgreSQL based on MVCC architecture.

What is Multi Version Concurrency Control (MVCC)

The day before yesterday, one of our developers asked a question like I want updated data on a table while a function is running in PostgreSQL.

If you are from Microsoft SQL Server background, of course, you will need this because you can get an updated data on a table while a function is running. Why? Because SQL Server is auto-committed by default.

When you create a function in PostgreSQL, you must put all your code between BEGIN and END. In PostgreSQL, it treats as transaction block, and you can get only updated table data once function execution completed or you can get the previous state of your table if execution of a function failed.

Why? because of MVCC architecture you cannot get partially updated table otherwise other users can update those value which breaks the rule of MVCC.

While function is running, a user can still access the old records of a table, and once function execution got completed, a user can access the newly updated records of a table.

CREATE FUNCTION fn_test()
  RETURNS as
$$
declare
	-- declare variables...
begin
	-- do operations...
end
$$ language plpgsql;

Then, they asked one more question like If the function code is default between BEGIN and END transaction block when we should define explicitly BEGIN TRANSACTION and COMMIT TRANSACTION in a PostgreSQL function?

My answer was, you can also set BEGIN TRANSACTION and COMMIT TRANSACTION explicitly when you want ROLLBACK or SAVEPOINT action.
But if you do not define this, by default everything is in transaction block and you will get all the output once your function completes its execution.

REATE FUNCTION fn_test()
  RETURNS as
$$
declare
	-- declare variables...
begin	
	-- do operations...
	begin transaction
		-- do transactions...
	commit transaction
end
$$ language plpgsql;

If you don’t know about the MVCC, please visit the below article.
Because without knowing the MVCC, you should not start the learning of PostgreSQL.

What is Multi Version Concurrency Control (MVCC)

Many times, I shared information on MVCC like ONLY INSERT concept where every update is a new version of data.

In today’s post, I am sharing a proof of MVCC.
Not in detail, but with the small and easy demonstration.

The XMIN is a pseudo column of PostgreSQL which used to track concurrent row access.
Using XMIN, you can get an exact number which internally uses by MVCC operations.

Please check the below demonstration:

Create a table with a sample record:

CREATE TABLE tbl_MVCCDemo (ID INT, CODE CHARACTER VARYING);

INSERT INTO tbl_MVCCDemo VALUES (1,'XVK98');

Check the value of XMIN pseudo column:

SELECT XMIN,ID,CODE
FROM tbl_MVCCDemo;

 xmin  | id  |  code
----------------------
  622  |  1  | XVK98

Now, update a record using XMIN:

UPDATE tbl_MVCCDemo SET code = 'T2738'
WHERE XMIN = 622;

Check the value of XMIN pseudo column:
We updated the value base on XMIN, then also the value of XMIN changed from 622 TO 623.
So this is proof that MVCC architecture inserts the new row for each update.

SELECT XMIN,ID,CODE
FROM tbl_MVCCDemo;

 xmin  | id  |  code
----------------------
  623  |  1  | XVK98

The Visibility Map is one of the important architectural notes of PostgreSQL.

The Visibility Map associates with each Table and Index.

It stores in separate relation alongside with main relation and It starts with the file node number plus a suffix _vm.

The Visibility Map uses to keep track of which pages contain only tuples that are known to be visible to all active transactions.

It stores one bit per page. If it set bit, that means no free space on the page and all tuples can be visible to running transaction.

If you do not know about the MVCC (Multiversion Concurrency Control), please visit below an article.

What is Multi Version Concurrency Control (MVCC)

Why VACUUM requires in MVCC because VACUUM removes dead row versions in tables and marks the space available for future reuse.

With the Visibility Map, VACUUM will be now able to perform partial scans of the table data, skipping pages which marks as fully visible.

Now Database Administrator happy because partial scans require fewer I/O operations for VACUUM.

The Visibility map bits are only set by vacuum but are cleared by any data-modifying operations on a page.

Which two isolation levels support optimistic/row-versioned-based concurrency control?

First is READ COMMITTED and Second is SNAPSHOT Isolation level.

What is the size of a lock structure?

The size of lock structure is 96 bytes.

Which isolation level is using key-range lock?

The SERIALIZABLE isolation level is using key-range lock.

Can deadlocks occur on resources other than a database object and if it YES, what are that other resources?

YES, deadlock can occur on other resources like: worker threads, Memory, Parallel query execution related resources, Multiple Active Result Sets resources.

When to use Lock Pages in Memory with SQL Server?

The Lock Pages in Memory user right is a good performance optimization practice when Tier 1 mission-critical SQL Server. When setting the SQL Server Lock Pages in Memory user right, the virtual machine’s memory reservation should also be set to match the amount of the provision memory.

What is the use of DEADLOCK_PRIORITY option?

SET DEADLOCK_PRIORITY is set at execute or run time and not at parse time.

If the sessions have different deadlock priorities, the session with the lowest deadlock priority is chosen as the deadlock victim.

What is the use of LOCK_TIMEOUT option?

It is the number of milliseconds that will pass before Microsoft SQL Server returns a locking error.

A value of -1 (default) indicates no time-out period and value of 0 means to not wait at all and return a message as soon as a lock is encountered.

How to check the value of LOCK_TIMEOUT?

SELECT @@LOCK_TIMEOUT
GO

How you can set maximum number of available locks to minimize the memory usage?

You can use sp_configure to set a locks value.

Below is a T-SQL Script to set the lock value up to 20000.

Use AdventureWorks2012;
GO
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'locks', 20000;
GO
RECONFIGURE;
GO

Which steps should be taken to address common deadlock issues?

• Understand which tables are involved in the deadlocks.
• Understand which pieces of code correspond to the deadlock and which users execute those pieces of code.
• Understand how the application is affected when a deadlock occurs.
• Determine the length of the transaction.
• Determine if the tables are highly fragmented and causing unnecessary page reads to complete the transaction.
• Determine if the tables can be redesigned to prevent deadlocking.
• Determine if additional or less indexes would improve the deadlocking.
• Determine if the code accesses the tables in the same order and with a similar query plan.

Can we capture deadlock information in the XML File format?

SQL Server Profiler has the ability to capture the deadlock related information as XML files which can be analyzed to determine the overall locking and blocking issue.

What are the features of XML Deadlock File?

• The XML Deadlocks feature gives you the ability to capture the deadlock related information and review the deadlocks independent of the Profiler interface.
• Each of the deadlock files can be analyzed in order to determine trends with the deadlocks on your SQL Server.
• Each deadlock situation can be stored in a separate file for per deadlock analysis.
• The XML format gives you the ability to programmatically review the deadlocks.

What is the Multiversion Concurrency Control (MVCC)?

What is Multi Version Concurrency Control (MVCC)

What is the Implicit Locking?

Implicit Lock means Lock is implied but is not acquired.

The Database will implicitly obtain the appropriate locks for your application at the point at which they are needed. An operation that reads an object will obtain a read lock; an operation that modifies an object will obtain a write lock.

What is the Explicit Locking?

Explicit Lock means Lock is explicitly requested for a record or table.

An application needing to reserve access to all required objects in advance can explicitly lock objects.

Explicit locking effectively freezes the objects, because no other session can modify them as long as they are locked.

I am confused between UPDLOCK and HOLDLOCK, would you please help me?

UPDLOCK does not block concurrent SELECT nor INSERT, but blocks any UPDATE or DELETE of the rows selected by T1.

HOLDLOCK means SERALIZABLE and therefore allows SELECTS, but blocks UPDATE and DELETES of the rows selected by T1, as well as any INSERT in the range selected by T1.

What is the meaning of wait and await in the Deadlock situation?

wait will synchronously block until the task completes. So the current thread is literally blocked waiting for the task to complete. As a general rule, you should use “async all the way down”; that is, don’t block on async code.

await will asynchronously wait until the task completes. This means the current method is “paused” (its state is captured) and the method returns an incomplete task to its caller. Later, when the await expression completes, the remainder of the method is scheduled as a continuation.

What is a Transaction in SQL Server ?

Transaction groups a set of T-Sql Statements into a single execution unit. Each transaction begins with a specific task and ends when all the tasks in the group successfully complete. If any of the tasks fails, the transaction fails. Therefore, atransaction has only two results: success or failure.

What are the types of Transactions?

Implicit – Specifies any Single Insert,Update or Delete statement as Transaction Unit.  No need to specify Explicitly.

Explicit – A group of T-Sql statements with the beginning and ending marked with Begin Transaction,Commit and RollBack. PFB an Example for Explicit transactions.