In this post, I am sharing a script to find a list of blocked and deadlock queries by other blocking queries of Greenplum Database Server.

You can visit below article to know more about, what is a lock, deadlock, block and timeout?

Database Theory: What is Timeout, Lock, Block, and Deadlock in a Database System?

If two queries are trying to get the same resource, both are depended on each other and generates block state first and then deadlock.

In the below script, you can find a list of blocked queries and blocker queries with a value of time age.
You can find a process_id of blocker query which we can kill for avoiding the deadlock situation.

SELECT
	kl.pid as blocking_pid
	,ka.usename as blocking_user
	,ka.current_query as blocking_query
	,bl.pid as blocked_pid
	,a.usename as blocked_user
	,a.current_query as blocked_query
	,to_char(age(now(), a.query_start),'HH24h:MIm:SSs') as age
FROM pg_catalog.pg_locks bl
JOIN pg_catalog.pg_stat_activity a
	ON bl.pid = a.procpid
JOIN pg_catalog.pg_locks kl
	ON bl.locktype = kl.locktype
	and bl.database is not distinct from kl.database
	and bl.relation is not distinct from kl.relation
	and bl.page is not distinct from kl.page
	and bl.tuple is not distinct from kl.tuple
	and bl.transactionid is not distinct from kl.transactionid
	and bl.classid is not distinct from kl.classid
	and bl.objid is not distinct from kl.objid
	and bl.objsubid is not distinct from kl.objsubid
	and bl.pid <> kl.pid
JOIN pg_catalog.pg_stat_activity ka
	ON kl.pid = ka.procpid
WHERE kl.granted and not bl.granted
ORDER BY a.query_start;

In this post, I am sharing a script to find long running queries or sessions with the lock information of Greenplum database server.

In the below script, you can find the age column which gives you total execution time of running sessions. You can also find query lock related details.

select 
	age(now(),pg_stat_activity.query_start) as "age",
	pg_stat_activity.datname, pg_stat_activity.usename,
	pg_stat_activity.client_addr, pg_stat_activity.application_name,
	pg_locks.transactionid, pg_locks.transaction,
	pg_stat_activity.procpid, pg_stat_activity.query_start,
	pg_locks.mode, pg_stat_activity.current_query
from  pg_stat_activity, pg_locks
where pg_locks.pid = pg_stat_activity.procpid
and procpid <> pg_backend_pid()
and pg_locks.transactionid = pg_locks.transaction
order by 1 desc;

I shared a couple of articles on SQL Server database locking and how to detect a deadlock. Please access few of them.

Different ways to detect a deadlock
What happens after the Deadlock?

I recently found an undocumented function %%lockres%% which we can use to locate the page information of records especially for those records which hold a Lock.

It returns a hash value same as like column “resource_description” of table sys.dm_tran_locks, but the good thing is you can use it in your SELECT statement and can correlate the column resource_description of table sys.dm_tran_locks.

By this way, you can get the exact record, and it’s page info which currently held by a Lock.

Let me demonstrate this,
Prerequisites:
Use below article and generate a sample deadlock situation in your database server.

SQL Server: T-SQL script to generate a DeadLock in a Database

Check the result of below query and note down the result of column “resource_description”:

select *from sys.dm_tran_locks

Use %%lockres%% and SELECT both the tables:
You can easily correlate the result of sys.dm_tran_locks column resource_description and the result of %%lockres%%. You can find an exact record which held by a Lock.

SELECT %%lockres%% AS RecordPageInfo,* 
FROM tbl_SampleTable_A

SELECT %%lockres%% AS RecordPageInfo,* 
FROM tbl_SampleTable_B

After this exercise, Please COMMIT all your open transactions.

The DeadLock in a database system is a very common thing and all Database Administrator responsible to detect and manage a DeadLock situation.

In this post, I am sharing one small demonstration t to generate a DeadLock situation in a PostgreSQL.

Database Theory: What is Timeout, Lock, Block, and Deadlock in a Database System?

Most of the time, We are facing DeadLock problem in our Production Database Server.
But What happen, when Database Administrator wants to generate DeadLock situation and wants to create a script for detecting a DeadLock.

First create two sample tables:

CREATE TABLE ABC (ID INT PRIMARY KEY);
CREATE TABLE XYZ (ID INT PRIMARY KEY);

Open Session – 1 and execute below code:

BEGIN;
INSERT INTO ABC VALUES (1);

Open Session – 2 and execute below code:

BEGIN;
INSERT INTO XYZ VALUES (1);
INSERT INTO ABC VALUES (1);

Again open Session – 1 and execute below Insert Statement:

INSERT INTO XYZ VALUES (1);

You will get a below deadlock message:

ERROR:  deadlock detected
DETAIL:  Process 5944 waits for ShareLock on transaction 892; blocked by process 4884.
Process 4884 waits for ShareLock on transaction 891; blocked by process 5944.
HINT:  See server log for query details.
CONTEXT:  while inserting index tuple (0,2) in relation "xyz_pkey"

Please visit other related articles:

PostgreSQL 9.5: Using FOR UPDATE SKIP LOCKED Option SELECT only Committed Records

PostgreSQL: Script to find information about the Locks held by Open Transactions (pg_locks)

Script to find sessions that are blocking other sessions in PostgreSQL

In this post, I am sharing one script to find information about the Locks which are held by Open Transactions of PostgreSQL Database Server.

Database Theory: What is Timeout, Lock, Block, and Deadlock in a Database System?

PostgreSQL view pg_locks contains the main information about the all types of lock of Database Server.
It contains one row per active lockable object.

This script is very helpful to all PostgreSQL Database Administrators.

Below is a script:

SELECT 
	pl.pid AS ProcessID
	,psa.datname AS DatabaseName
	,psa.usename AS UserName
	,psa.application_name AS ApplicationName
	,ps.relname AS ObjectName
	,psa.query_start AS QueryStartTime
	,psa.state AS QueryState
	,psa.query AS SQLQuery
	,pl.locktype
	,pl.tuple AS TupleNumber
	,pl.mode AS LockMode
	,pl.granted -- True if lock is held, false if lock is awaited
FROM pg_locks AS pl 
LEFT JOIN pg_stat_activity AS psa
	ON pl.pid = psa.pid
LEFT JOIN pg_class AS ps
	ON pl.relation = ps.oid

PostgreSQL 9.5 introduced one of the powerful feature called SKIP LOCKED which is used for SKIP uncommitted data.

I am not going to make this more complex topic, explaining by one practical example.

In our project we have implemented thread mechanism. Different threads are running and taking a fixed set of records from the Database.
These threads basically implemented for performing Reverse Geo Coding on the stored Latitude and Longitude.
I am giving 10000 records to each thread in each batch.
Now the average execution time of each thread is 5 second to 7 second.

Now, one batch of 10000 records taken by thread_1 for 5 second to 7 second that same records should not taken by any other thread because that first 10000 records already under the process by thead_1.
Thread_2 has taken next 10000 records which are not taken by thread_1.

Now with the PostgreSQL 9.5, we do not require any additional locking mechanism or isolation level to achieve this scenario.
We can very easily achieve using FOR UPDATE SKIP LOCKED option. When we SELECT data using this option, it will skip all running transactions (uncommitted transaction) and return only required records to be processed.

Below is a full demonstration of this.

First create a table with sample data:

CREATE TABLE tbl_Employee 
(
	EmpID INT
	,EmpName CHARACTER VARYING
);

INSERT INTO tbl_Employee 
VALUES (1,'Anvesh'),(2,'Roy')
,(3,'Benny'),(4,'Nivu');

Now we are going to test this option with the help of two different sessions.

Open a new session and execute below queries in Session 1:
We have inserted four sample records and now will update one of record in OPEN TRANSACTION with out committing anything.

BEGIN TRANSACTION

UPDATE tbl_Employee SET EmpName = 'Martin' 
WHERE EmpID = 3;

Open another session and execute below queries in Session 2:
First SELECT with out any option:

If you do not know, please read about the MVCC.

What is Multi Version Concurrency Control (MVCC)

PostgreSQL has a default MVCC architecture, in which you can select last committed row version.
In the above query we have updated one record for EmpID=3, but when I execute simple SELECT statement then it returned the old value of EmpID=3.

This data is based on last committed row version.

SELECT *FROM tbl_Employee;

/* Result:
EmpID    EmpName
--------------------
  1      Anvesh
  2      Roy
  4      Nivu
  3      Benny
*/

Now SELECT with NOWAIT option:

When you are trying to SELECT running transaction, It will raise the error like : can not obtain lock on row. You can not SELECT EmpID=3, but you can SELECT EmpID IN (1,2,4).

Please find out below query and the result.

SELECT *FROM tbl_Employee WHERE EmpID=3 FOR UPDATE NOWAIT;
SELECT *FROM tbl_Employee FOR UPDATE NOWAIT;

/* Result:
********** Error **********

ERROR: could not obtain lock on row in relation "tbl_employee"
SQL state: 55P03
*/

Now finally, we test SKIP LOCKED option:

A SKIP LOCKED option simply sikped the records or rows which are the part of ROW Level Locking.
You can find out in the below result, EmpID=3 is not in result because it is not committed yet and running in different session.

/* Result:
SELECT *FROM tbl_Employee FOR UPDATE SKIP LOCKED;

EmpID    EmpName
--------------------
  1      Anvesh
  2      Roy
  4      Nivu
*/

Which two isolation levels support optimistic/row-versioned-based concurrency control?

First is READ COMMITTED and Second is SNAPSHOT Isolation level.

What is the size of a lock structure?

The size of lock structure is 96 bytes.

Which isolation level is using key-range lock?

The SERIALIZABLE isolation level is using key-range lock.

Can deadlocks occur on resources other than a database object and if it YES, what are that other resources?

YES, deadlock can occur on other resources like: worker threads, Memory, Parallel query execution related resources, Multiple Active Result Sets resources.

When to use Lock Pages in Memory with SQL Server?

The Lock Pages in Memory user right is a good performance optimization practice when Tier 1 mission-critical SQL Server. When setting the SQL Server Lock Pages in Memory user right, the virtual machine’s memory reservation should also be set to match the amount of the provision memory.

What is the use of DEADLOCK_PRIORITY option?

SET DEADLOCK_PRIORITY is set at execute or run time and not at parse time.

If the sessions have different deadlock priorities, the session with the lowest deadlock priority is chosen as the deadlock victim.

What is the use of LOCK_TIMEOUT option?

It is the number of milliseconds that will pass before Microsoft SQL Server returns a locking error.

A value of -1 (default) indicates no time-out period and value of 0 means to not wait at all and return a message as soon as a lock is encountered.

How to check the value of LOCK_TIMEOUT?

SELECT @@LOCK_TIMEOUT
GO

How you can set maximum number of available locks to minimize the memory usage?

You can use sp_configure to set a locks value.

Below is a T-SQL Script to set the lock value up to 20000.

Use AdventureWorks2012;
GO
sp_configure 'show advanced options', 1;
GO
RECONFIGURE;
GO
sp_configure 'locks', 20000;
GO
RECONFIGURE;
GO

Which steps should be taken to address common deadlock issues?

• Understand which tables are involved in the deadlocks.
• Understand which pieces of code correspond to the deadlock and which users execute those pieces of code.
• Understand how the application is affected when a deadlock occurs.
• Determine the length of the transaction.
• Determine if the tables are highly fragmented and causing unnecessary page reads to complete the transaction.
• Determine if the tables can be redesigned to prevent deadlocking.
• Determine if additional or less indexes would improve the deadlocking.
• Determine if the code accesses the tables in the same order and with a similar query plan.

Can we capture deadlock information in the XML File format?

SQL Server Profiler has the ability to capture the deadlock related information as XML files which can be analyzed to determine the overall locking and blocking issue.

What are the features of XML Deadlock File?

• The XML Deadlocks feature gives you the ability to capture the deadlock related information and review the deadlocks independent of the Profiler interface.
• Each of the deadlock files can be analyzed in order to determine trends with the deadlocks on your SQL Server.
• Each deadlock situation can be stored in a separate file for per deadlock analysis.
• The XML format gives you the ability to programmatically review the deadlocks.

What is the Multiversion Concurrency Control (MVCC)?

What is Multi Version Concurrency Control (MVCC)

What is the Implicit Locking?

Implicit Lock means Lock is implied but is not acquired.

The Database will implicitly obtain the appropriate locks for your application at the point at which they are needed. An operation that reads an object will obtain a read lock; an operation that modifies an object will obtain a write lock.

What is the Explicit Locking?

Explicit Lock means Lock is explicitly requested for a record or table.

An application needing to reserve access to all required objects in advance can explicitly lock objects.

Explicit locking effectively freezes the objects, because no other session can modify them as long as they are locked.

I am confused between UPDLOCK and HOLDLOCK, would you please help me?

UPDLOCK does not block concurrent SELECT nor INSERT, but blocks any UPDATE or DELETE of the rows selected by T1.

HOLDLOCK means SERALIZABLE and therefore allows SELECTS, but blocks UPDATE and DELETES of the rows selected by T1, as well as any INSERT in the range selected by T1.

What is the meaning of wait and await in the Deadlock situation?

wait will synchronously block until the task completes. So the current thread is literally blocked waiting for the task to complete. As a general rule, you should use “async all the way down”; that is, don’t block on async code.

await will asynchronously wait until the task completes. This means the current method is “paused” (its state is captured) and the method returns an incomplete task to its caller. Later, when the await expression completes, the remainder of the method is scheduled as a continuation.

What is a Transaction in SQL Server ?

Transaction groups a set of T-Sql Statements into a single execution unit. Each transaction begins with a specific task and ends when all the tasks in the group successfully complete. If any of the tasks fails, the transaction fails. Therefore, atransaction has only two results: success or failure.

What are the types of Transactions?

Implicit – Specifies any Single Insert,Update or Delete statement as Transaction Unit.  No need to specify Explicitly.

Explicit – A group of T-Sql statements with the beginning and ending marked with Begin Transaction,Commit and RollBack. PFB an Example for Explicit transactions.

Database session A tries to update some data that is already locked by database session B. What happens to session A?

The session A will actually be placed in what’s called a lock wait state, and session A will be stopped from making further progress with any SQL transaction that it’s performing. Another way of saying this is that session A will be “stalled” until session B releases the lock on that data.

What is lock contention?

One problem that occurs with having locks is that locks can cause what’s known as contention.

In the best case, lock contention means that some user processes run slower because a session is waiting for a lock. In the worst case, having sessions compete for locks can make sessions stall for an indefinite period of time.

What is Locking Hints option?

Locking hints can be specified for individual table references in the SELECT, INSERT, UPDATE, and DELETE statements. The hints specify the type of locking or row versioning the instance of the SQL Server Database Engine uses for the table data.

Does Locking hints override the current transaction isolation level?

Yes, The Locking hints override the current transaction isolation level for the session.

What is WITH (NOLOCK)?

It is equivalent to READUNCOMMITTED isolation level and you can SELECT table dirty data which are not yet committed.

What is WITH (NOWAIT)?

It instructs the Database Engine to return a message as soon as a lock is encountered on the table. NOWAIT is equivalent to specifying SET LOCK_TIMEOUT 0 for a specific table.

What is WITH (ROWLOCK)?

It specifies that row locks are taken when page or table locks are ordinarily taken.

What is WITH (TABLOCK)?

Specifies that the acquired lock is applied at the table level. The type of lock that is acquired depends on the statement being executed.

What is WITH (UPDLOCK)?

It specifies that update locks are to be taken and held until the transaction completes. UPDLOCK takes update locks for read operations only at the row-level or page-level.

What is the use of sys.dm_tran_locks?

You can query the sys.dm_tran_locks dynamic management view to obtain information about the current state of locking in an instance of the Database Engine.

What is a Lock Escalation?

A query might have a resource locked for “shared” and escalate it to “exclusive” to perform an update. SQL Server does Lock Escalation in order to reduce memory overhead, by converting several fine-grained low level locks to coarse-grained high level locks. Page Locks are also escalated to Table Locks.

What is the Spinlock?

It controls access to a lock resource, such as a row or a table.

Without lock partitioning, one spinlock manages all lock requests for a single lock resource.

To reduce contention on a single lock resource, lock partitioning splits a single lock resource into multiple lock resources to distribute the load across multiple spinlocks.

How READ_COMMITTED internally implemented row level locks?

When SQL Server executes a statement at the read committed isolation level, it acquires short lived share locks on a row by row basis. The duration of these share locks is just long enough to read and process each row; the server generally releases each lock before proceeding to the next row.

Within the REPEATABLE_READ and SERIALIZABLE isolation levels, during a read operation and assuming row-level locking, how long are locks held or retained for?

Answer: Within either of these isolation levels, locks are held for the duration of the transaction, unlike within the READ_COMMITTED isolation level.

When Lock escalation is triggered?

When a single Transact-SQL statement acquires at least 5,000 locks on a single table or index.

When the number of locks in an instance of the Database Engine exceeds memory or configuration thresholds.

If locks cannot be escalated because of lock conflicts, the Database Engine periodically triggers lock escalation at every 1,250 new locks acquired.

Name as many of the lockable resources of the SQL Server.

• RID (single row on a heap)
• KEY (single row (or range) on an index)
• PAGE
• EXTENT
• HOBT (heap or b-tree)
• TABLE (entire table, all data and indexes)
• FILE
• APPLICATION
• METADATA
• ALLOCATION_UNIT
• DATABASE

What is the least restrictive and the most restrictive type of lock?

The least restrictive type of lock is a shared lock. The most restrictive type of lock is a schema-modification.

What is the Read Uncommitted Isolation Level of the SQL Server?

SQL Server: What is Read Uncommitted Isolation Level?

What is the READ COMMITTED Isolation Level of the SQL Server?

SQL Server: READ COMMITTED Isolation Level with the READ_COMMITTED_SNAPSHOT option

What is the READ COMMITTED Isolation Level with the READ_COMMITTED_SNAPSHOT option?

When the READ_COMMITTED_SNAPSHOT database option is ON, read committed isolation uses row versioning.

READ COMMITTED SNAPSHOT does optimistic reads and pessimistic writes. When data is updating, you can still read old version of data and there is no any share lock, but you cannot update that same data until the running update is not finished.

What is the REPEATABLE READ Isolation Level of the SQL Server?

SQL Server: What is REPEATABLE READ Isolation Level?

What is the SERIALIZABLE Isolation level of the SQL Server?

SQL Sever: What is SERIALIZABLE Isolation level?

What is the SNAPSHOT Isolation Level of the SQL Server?

SQL Server: What is SNAPSHOT Isolation Level?

Can you enable SNAPSHOT Isolation Level for a particular SQL statement?

You cannot enable SNAPSHOT Isolation level at statement level.

You have to enable at the database level and after enabling all your database transactions will execute in MVCC fashion.

What is the difference between READ_COMMITTED_SNAPSHOT option and Snapshot Isolation Level?

Using READ_COMMITTED_SNAPSHOT option, you can SELECT old version of rows while it is updating, but you cannot execute new UPDATE on the same data until it finished a running UPDATE.

Using SNAPSHOT ISOLATION LEVEL, you can perform any kind of DML operation because it is managing different data version in the TempDB so the reader and writer never blocked by each other.

What is a Shared lock?

Shared (S): Used for read operations that do not change or update data, such as a SELECT statement.

What is an Update lock?

Update(U): Used on resources that can be updated. Prevents a common form of deadlock that occurs when multiple sessions are reading, locking, and potentially updating resources later.

What is an Exclusive lock?

Exclusive(X): Used for data-modification operations, such as INSERT, UPDATE, or DELETE. Ensures that multiple updates cannot be made to the same resource at the same time.

What is an Intent lock?

The Database Engine uses intent locks to protect placing a shared (S) lock or exclusive (X) lock on a resource lower in the lock hierarchy. Intent locks are named intent locks because they are acquired before a lock at the lower level, and therefore signal intent to place locks at a lower level.

What is a Schema lock?

The Database Engine uses schema modification (Sch-M) locks during a table data definition language (DDL) operation, such as adding a column or dropping a table. During the time that it is held, the Sch-M lock prevents concurrent access to the table.

What is a Bulk Update lock?

BulkUpdate (BU): Used when bulk copying data into a table and the TABLOCK hint is specified.

What is a Key range lock?

Key-range locks protect a range of rows implicitly included in a record set being read by a Transact-SQL statement while using the serializable transaction isolation level. Key-range locking prevents phantom reads.

A key range lock protects this requirement by preventing other transactions from inserting new rows whose keys would fall in the range of keys read by the serializable transaction.

Do you think lock is reducing the performance of the database system?

For large computer systems, locks on frequently referenced objects can become a performance bottleneck as acquiring and releasing locks place contention on internal locking resources.

What do you know about the lock partitioning?

Lock partitioning enhances locking performance by splitting a single lock resource into multiple lock resources. This feature is only available for systems with 16 or more CPUs, and is automatically enabled and cannot be disabled. Only object locks can be partitioned.

Timeout:

When the client application is waiting for a require resource, Timeout occurs.

There are multiple reasons for Timeout like transaction blocking, the distance network, high CPU utilization and multiple request user request at one time.

The Database Timeout cannot affect to other running connection, and it treated independently, but for solving the Timeout issue we need to check other causes like OS response, Network response, Database Timeout threshold value, number of Database Connections and other.

Lock:

When one process is accessing an object, it can request for a lock for that particular object so that other processes cannot perform read or update on that object.

There are different types of lock like Exclusive Lock, Non-exclusive Lock.

An Exclusive Lock prevents object or data from the reading and writing both the actions.
A Non-Exclusive Lock allows read-only access for an object or data, but it prevents the write action for an object or data.

Block:

If two processes are accessing the same resource so at that time one process has to wait until the other process releases the occupied resource.

This is called blocking because one transaction is blocked by another transaction and all blocked processes kept in the waiting queue.

If the blocked transaction is waiting for a long time, it may generate a Database Timeout.
If the long-running blocked transaction requires a lock, it may generate a Database Deadlock.

Deadlock:

One X process is waiting to complete the transaction of the Y process, and the same time, Y process is blocked because it is also waiting to complete the transaction of the X process.
This is called as Deadlock situation.

In the Deadlock situation, both the processes trying to access a particular resource and also has a lock on that resource by each other, so both are waiting for each other to release a lock and complete the transaction.

Deadlock is a next stage of the blocking situation so if you want to avoid Deadlock situation, you should first find the blocked transaction and tried to remove internal dependency for those transactions.