Whenever you are searching for SQL query optimization tips or prevent SQL injection tips, you can find few common suggestions like: Please do not use Dynamic SQL in your Stored Procedure. Please try to avoid the use of Temp Tables.

I always tell to database developers, please try to avoid the use of Dynamic SQL and Temp Table in your Stored Procedures.

A Temp Table requires additional CPU/IO which degrades the query performance. A Dynamic SQL is not safe against the SQL Injection and It also compiles data every time which requires more Memory.

Here, I am sharing one of the important scripts which gives you list of SQL Server Stored Procedures, which use the Dynamic SQL and Temp Table.
You can easily get the list of Stored Procedures and can make plans of modifying it.

Prepared this script using sys.dm_exec_describe_first_result_set and sys.procedures where error number 4 = Dynamic SQL and error number 10 = Temporary Table.

SELECT  
	SP_Name
	,error_message
	,error_type_desc 
FROM (SELECT schema_name(schema_id)+'.'+name SP_Name FROM sys.procedures) AS T
CROSS APPLY sys.dm_exec_describe_first_result_set (SP_Name, NULL, 0) 
WHERE error_type IN (4,10)

The Database Administrator must know something about the best configuration of TempDB.
A general advice is, try to avoid the use of TempDB by creating temp tables.

But TempDB also stores other information related to query execution, internal sort operation, overflow memory data.

You should keep your eye open for TempDB otherwise it degrades the overall performance.

SQL Server: The TempDB is Full, Shrink it or Move it

Sharing few tips to improve the TempDB performance:

Try to keep TempDB in local drive:

You should store TempDB in your local SSD drive instead of the shared network drive. This introduced by SQL Server 2012.

If you store other databases in shared network drive and TempDB in local drive, It creates a separate path for IO which divides the traffic and improves the performance.

Should configure multiple DATA Files:

You should create more data files equal to 1/2 or 1/4 the physical CPUs.

Once you have attached different data files to it, it stripped the files and performed the parallel operations.
Now It has its own PFS, GAM, and SGAM pages, so it spreads the writing process which improves the writing performance.

Few other tips:

• Set the recovery model of tempdb to SIMPLE.
• Allow for tempdb files to automatically grow as required.
• Set preallocate space for all tempdb files and it prevents tempdb from expanding too frequently.
• Make each data file the same size.

Anuraag Veerapaneni – He is my work coordinator, and he got “OID does not exist” error in one of the function execution.

Later we found that the problem was in the function code. The temp table was creating and dropping within the WHILE LOOP.

Internally, WHILE LOOP also keeps one session, so the OBJECT ID of dropped temp table still there in the session of WHILE LOOP.
After dropping the temp table, it creates a new temp table in WHILE LOOP with the new object id but dropped temp table object id is still in the session so while selecting a temp table it will search for old Temp table which already dropped.

Our advice: please never write code to create or drop temp tables in the WHILE LOOP.

For better understanding, please visit below demonstration:

Create a test function:

CREATE OR REPLACE FUNCTION fn_check_drop_temp_table()
RETURNS integer
AS
$dbrnd$
 
DECLARE i INT;
j OID;
z OID;
b OID;
a OID;
 
BEGIN   
    i:=0;   
    z:=Coalesce((SELECT relfilenode FROM pg_class
    WHERE relname LIKE '%tmp_drop_test%'),111111);
   
    RAISE NOTICE '%','No table found...'|| z;
   
    DROP TABLE IF EXISTS tmp_a;
    CREATE TABLE tmp_a (col1 int);
       
    WHILE i<=2
    LOOP   
		j:=coalesce((SELECT relfilenode FROM pg_class
		WHERE relname LIKE '%tmp_drop_test%'),1);
		
		RAISE NOTICE '%','1= No data, Other value= OID Data --'|| j;
	   
		DROP TABLE IF EXISTS tmp_drop_test; -- Now drop temp table
		CREATE temp TABLE tmp_drop_test AS SELECT 1 AS int1; -- Create temp table again with same name
	   
		a:=coalesce((SELECT relfilenode FROM pg_class
		WHERE relname LIKE '%tmp_drop_test%'),1);
	   
		RAISE NOTICE '%','After creating temp table - OID of temp table:'|| a; 
		
		INSERT INTO tmp_a
		SELECT * FROM tmp_drop_test;
						
		i:=i+1;   
    END LOOP;   
                RETURN 1;
END;
$dbrnd$
LANGUAGE PLPGSQL;

Now, execute above function:

SELECT *FROM fn_check_drop_temp_table();

Got error in the result:

NOTICE:  No table found...111111
 
NOTICE:  1= No data, Other value= OID Data --1
 
NOTICE:  After creating temp table - OID of temp table:27102093
 
NOTICE:  1= No data, Other value= OID Data --27102093
 
NOTICE:  After creating temp table - OID of temp table:27102120
 
 
ERROR:  relation with OID 27102093 does not exist
CONTEXT:  SQL statement "INSERT INTO tmp_a SELECT * FROM tmp_drop_test"
PL/pgSQL function "fn_check_drop_temp_table" line 33 at SQL statement

In this post, I am sharing a script to find a list of Temp tables with their size and user information in PostgreSQL.

We all aware of the temp tables and many times developers are creating temp table for their ad-hoc testing purpose.
But unfortunately sometimes, they forget to drop unused temp tables, and temp tables keep active in those idle connections. It also requires unnecessary utilization of disk space.

So now, DBAs are responsible for identifying unused temp tables and then inform to developers so that they can drop their unused temp tables.

SELECT
	n.nspname as SchemaName
	,c.relname as RelationName
	,CASE c.relkind
	WHEN 'r' THEN 'table'
	WHEN 'v' THEN 'view'
	WHEN 'i' THEN 'index'
	WHEN 'S' THEN 'sequence'
	WHEN 's' THEN 'special'
	END as RelationType
	,pg_catalog.pg_get_userbyid(c.relowner) as RelationOwner               
	,pg_size_pretty(pg_relation_size(n.nspname ||'.'|| c.relname)) as RelationSize
FROM pg_catalog.pg_class c
LEFT JOIN pg_catalog.pg_namespace n               
                ON n.oid = c.relnamespace
WHERE  c.relkind IN ('r','s') 
AND  (n.nspname !~ '^pg_toast' and nspname like 'pg_temp%')
ORDER BY pg_relation_size(n.nspname ||'.'|| c.relname) DESC

I have given one assignment to a database developer team for preparing general database development practices.
I have found one point in that document: “Do not create Indexes on Temp Tables”.

The use of Temp Tables is very common in SQL Server. Most of the SQL database developers are storing their data into temp tables.
Let me, explore something about, the best practices of Temp Tables.

In general advice, we should avoid the use of temp tables because it requires more CPU/IO while accessing the TempDB.
When you are storing data into temp tables, it stores in TempDB files means the physical disk operation is involved.

Like any other regular table, temp table also requires disk space to store temporary data and other resources for manipulation.

I have seen that many developers are saving a significant number of records in Temp Tables and they are not thinking even for Indexing on Temp Tables.

If you don’t know, please note: You can create a Clustered and Non-Clustered Indexes on Temp Tables which improves your query performance.

Again, more indexes will create more problems because it needs space and manipulation.
First, you should analyze the volume of data, execution plan and accordingly you should apply indexes on Temp tables.

Personally, I have checked the performance of Temp Tables with Clustered Index, which is much better than Temp Table without an Indexes.

I am rejecting all comments and points which say that ” we should not apply Indexes on Temp Tables. “

SQL Server: Difference between Temporary Table and Table Variable

MySQL 5.7 introduced INNODB_TEMP_TABLE_INFO table added into INFORMATION_SCHEMA.

Using INNODB_TEMP_TABLE_INFO, you can get A list of all active Temp Table of MySQL InnoDB Instance.
You can get the information of all active user created and system created Temp Tables.

The Database Administrator can use this script for monitoring total Temp Table in MySQL Server.
Unnecessarily, we should not create a TEMP Table because It requires temporary disk + memory storage and degrade the overall performance of MySQL Server.

Once you get the list of tables using INNODB_TEMP_TABLE_INFO, you can get system generated Temp Table Name.

Check existance of INNODB_TEMP_TABLE_INFO:

SHOW TABLES FROM INFORMATION_SCHEMA LIKE 'INNODB_TEMP%';

Create two sample Temporary Table:

CREATE TEMPORARY TABLE ABC (ID INT, Name VARCHAR(50));
CREATE TEMPORARY TABLE XYZ (ID INT, Name VARCHAR(50));

Get the information of Temp Table:

SELECT * FROM INFORMATION_SCHEMA.INNODB_TEMP_TABLE_INFO

*************************** 1. row ***************************
            TABLE_ID: 93
                NAME: #sqlcec_9_bec
              N_COLS: 5
               SPACE: 76
PER_TABLE_TABLESPACE: FALSE
       IS_COMPRESSED: FALSE
*************************** 2. row ***************************
            TABLE_ID: 92
                NAME: #sqlcec_9_beb
              N_COLS: 5
               SPACE: 76
PER_TABLE_TABLESPACE: FALSE
       IS_COMPRESSED: FALSE
2 rows in set (0.01 sec)

What is an Inner join?

The join that displays only the rows that have a match in both the joined tables is known as inner join.

What is an Equi join?

Equi join is a special type of join in which we use only equality operator. Hence, when you make a query for join using equality operator, then that join query comes under Equi join.

Equi join has only (=) operator in join condition.

Equi join can be inner join, left outer join, right outer join.

What is a Left Outer join?

Left join displays all the rows from the first table and matched rows from second table like that.

What is a Right outer join?

Right outer join displays all the rows of second table and matched rows from first table.

What is a Full outer join?

Full outer join returns all the rows from both tables whether it has been matched or not.

What is a Cross Join?

A cross join that produces Cartesian product of the tables that are involved in the join. The size of a Cartesian product is the number of the rows in the first table multiplied by the number of rows in the second table.

What is a Self join?

Joining the table itself called self join. Self join is used to retrieve the records having some relation or similarity with other records in the same table. Here, we need to use aliases for the same table to set a self join between single table and retrieve records satisfying the condition in where clause

What is the difference between cross join and Full outer join?

Cross Join:

• It produces the Cartesian product.
• Results in Cartesian product of two tables.
• Results in pairs of rows.
• No join conditions are specified.

Full Outer Join:

• It includes all the rows from both the tables.
• Assigns NULL for unmatched fields.
• Results in every row from both of the tables , at least once.
• A combination of both left and right outer joins.

What is the difference between Temp table and Common table Expression (CTE)?

SQL Server: Difference between Temp Table and Common Table Expression (CTE)

What is the difference between Temp table and Table Variable?

SQL Server: Difference between Temporary Table and Table Variable

What is the difference between Local and Global Temp Table?

There are two types of temporary tables: local and global. Local temporary tables are visible only to their creators during the same connection to an instance of SQL Server as when the tables were first created or referenced. Local temporary tables are deleted after the user disconnects from the instance of SQL Server. Global temporary tables are visible to any user and any connection after they are created, and are deleted when all users that are referencing the table disconnect from the instance of SQL Server.

How to execute Large SQL Script File by using SQLCmd?

sqlcmd -S [servername] -d [databasename] -i [scriptfilename]

What is ASCII method?

ASCII will return the ascii value of the character expression.

Would do you please write syntax of table variable?

DECLARE @TableVariabl Table (ID INT, Name VARCHAR(150))

How do we unlock particular user in SQL Server?

First login with system admin user and execute below ALTER command.

ALTER LOGIN login_name WITH password=’password’ unlock;

What is the performance difference between EXISTS and IN?

EXISTS is efficient as if any row returned and it stops the further search.

IN is looking for a specific match which takes longer.

However, they may not always replace each other and we should use as per our system requirements.

Is SQL Case sensitive?

No: SQL is not case sensitive.

What are the different types of cardinality in SQL?

High-cardinality: refers to columns with values that are very uncommon or unique. High-cardinality column values are typically identification numbers, email addresses, or user names. An example of a data table column with high-cardinality would be a USERS table with a column named USER_ID. This column would contain unique values of 1-n. Each time a new user is created in the USERS table, a new number would be created in the USER_ID column to identify them uniquely. Since the values held in the USER_ID column are unique, this column’s cardinality type would be referred to as high-cardinality.

Normal-cardinality: refers to columns with values that are somewhat uncommon. Normal-cardinality column values are typically names, street addresses, or vehicle types. An example of a data table column with normal-cardinality would be a CUSTOMER table with a column named LAST_NAME, containing the last names of customers. While some people have common last names, such as Smith, others have uncommon last names. Therefore, an examination of all of the values held in the LAST_NAME column would show “clumps” of names in some places (e.g. a lot of Smith’s) surrounded on both sides by a long series of unique values. Since there is a variety of possible values held in this column, its cardinality type would be referred to as normal-cardinality.

Low-cardinality: refers to columns with few unique values. Low-cardinality column values are typically status flags, Boolean values, or major classifications such as gender. An example of a data table column with low-cardinality would be a CUSTOMER table with a column named NEW_CUSTOMER. This column would contain only two distinct values: Y or N, denoting whether the customer was new or not. Since there are only two possible values held in this column, its cardinality type would be referred to as low-cardinality.

How to find any Text, Column, Comment, Object Name in SQL Server Objects?

How to find any Text, Column, Comment, Object Name in SQL Server Objects

How to Find Missing Number and ID in the SQL Server?

Query to Find Missing Number and ID in SQL Server

In this post, I am sharing the important difference between Temp Table and Common Table Expression (CTE) of the SQL Server.
I recently wrote an article on the difference between Temp Table and Table Variable of the SQL Server.

SQL Server: Difference between Temporary Table and Table Variable

The theory and practice of Temp Table, Table Variable and CTE is one of my favourite topics of the SQL Server.

A CTE is used for a temporary result set that is defined within the execution scope of the query.
A Temp Table is also used for a temporary result set, but it can be defined for limited execution scope or can be used to define for global execution scope as a Global Temp Table.

A CTE is substituted for a view when the general use of a view is not required, and you do not have to store the definition of metadata.

CTE always uses memory and Temp Table stored in the TempDB.

The Scope of CTE is only for the query and can be referenced multiple times in other CTEs.
The child objects also access the Temp Table, and it persists until the end of the sessions.

We can not use NOLOCK and other isolation levels with CTE and with the Temp Table, we can apply it explicitly.

If we require to CTE for another batch query, it requires recreation of it.
The Temp Table can be used for all batches of a session.

A CTE doesn’t maintain any statistics and any metadata information, Temp table is maintaining a require statistics.

A CTE is a never part of the transaction and locking and its treated as system transaction.
The temp table operations are a part of user transactions.

If we have a large dataset, we should use Temp Table and with the small dataset, we can use CTE very effectively.

Sharing one of the timeworn topics and we can also find an N number of good articles on this.

I also like this topic very much so I would like to write a small note on this.

First, Very important point:

Temp table is stored in TempDB and Table variable is stored in Memory.
The above statement is 100% wrong.
Table variable is also stored definition into TempDB and if the data volume is increased, sometimes it also stores data into TempDB.

For that reason, SQL Server 2014 introduced Hekaton storage engine for Memory optimized Table.
"SQL Server 2014: What is Hekaton?"
"SQL Server 2014: Create Memory Optimized File Group and Table"

If we require creating a unique index on data, we should use Temporary Table. We cannot apply Index on Table Variable.

If we are dealing with large number records for inserting and deleting, we should use a temporary table. If we have a small quantity of data, we should use Table Variable.

If the amount of data is frequently changing, query planner requires updated statistics to recompile object so for that we should use Temporary Table.
If the amount of data is not frequently changing, we should use Table Variable for better performance.

When we require rollback because of an outer user transaction, we should use Table Variable because it automatically discard all the data from the session.

If we would like to avoid locking issue, we should go with Table Variable because Temp Table is fully dependent on TempDB so sometimes it holds the transactions.

We can define explicit transaction on Temp Table where with the Table Variable we cannot define explicit transaction.

The maintenance and creation of metadata for Table Variable requires less time than Temp Table.

The scope of Table Variable is up to batch or stored procedure after completion of execution it drops automatically.
The scope of Temp Table is up to sessions, and once the session ends, it drops automatically, or we can also drop explicitly.