As per the Microsoft, Use the cost threshold for parallelism option to specify the threshold at which Microsoft SQL Server creates and runs parallel plans for queries.

The Cost Threshold for Parallelism parameter is very important for Query Optimizer to prepare the query execution plan.

I wrote a title like Not Good – Because the default value of Cost Threshold for Parallelism is 5 which is very less now a day.

The value you define here which is for query optimizer where it finds a cheaper parallel execution plan, and if it fails to find parallel execution plan in given cost, it chooses the serial execution plan which degrades the query performance.

Now a day, we are dealing with millions/billions of data, so parallel query execution plan is good for big/complex queries so the default value 5 which is a tiny number.

The parallel execution plans will give you more suggestion on missing indexes which we can take for further query optimization.

I would suggest to change the value of Cost Threshold for Parallelism from 5 to 50 or even more.

How to change?

Using Script:

sp_configure 'show advanced options', 1;
GO
reconfigure;
GO
sp_configure 'cost threshold for parallelism', 50;
GO
reconfigure;
GO

Using SSMS:

For all the RDBMS, Database Statistics are playing the main role to generate an accurate Query Execution Plan.
A Database Administrator is also very much responsible to make sure that Database Statistics is updated.

MySQL query optimizer is also preparing Query Execution Plan with the help of Statistics Information.
The auto-update Statistics configuration is also very important to keep Statistics updated.

How to enable auto-update Statistics for InnoDB engine?

We can configure innodb_stats_on_metadata variable to enable auto-update Statistics.

When this variable is enabled InnoDB updates statistics when metadata statements such as SHOW TABLE STATUS or SHOW INDEX are run, or when accessing the INFORMATION_SCHEMA.TABLES or INFORMATION_SCHEMA.STATISTICS tables.

Check innodb_stats_on_metadata is ON or OFF:

SHOW VARIABLES LIKE 'innodb_stats_on_metadata';

If innodb_stats_on_metadata is OFF, make it ON:

SET GLOBAL innodb_stats_on_metadata=ON;

When you perform command like SHOW INDEX, Statisitcs will be update.
Accuracy of statistics is depends on MySQL Server Load.

SHOW INDEXES FROM SchemaName.TableName;

You can use below query to check the Index Statistics:

SELECT 
    Table_Schema
    ,Table_Name
    ,Non_Unique
    ,Index_Schema
    ,Seq_In_Index
    ,Column_Name
    ,Collation
    ,Cardinality
    ,Index_Type
FROM information_schema.statistics 
WHERE TABLE_SCHEMA='SchemaName';

The Storage Engine and Query Processor are both important core part of SQL Server. Storage Engine is responsible for data integrity and data storage in disk / memory while the Query Processor simply process the query bases on execution plan and return the result.

Most of the interviewer asks this question to SQL Server DBA that How SQL Server executes SQL Query and Statement so I have decided to prepare a short note on this.

How SQL Server internally executes SQL Query and Statement?

Parser:

The Parser makes sure that the T-SQL query has a valid syntax, and translates the SQL query into an initial logical tree representation with different information such as operators, join conditions, expressions, identifiers.
It sents this logical tree to Query Optimizer.

Binding:

It is a name resolution process and SQL Server makes sure that all object names are valid and they exist on a disk.
It creates algebrized tree with table and column names information and sents to Query Optimizer.

The Query Optimizer:

It uses logical tree and algebrized tree generated by Parsing and Binding Process and extracts the different information like, table data, table definition, database information and other database statistics.

If we look at specific table data extraction operation, there are different approaches like: selecting few records require Index Scan, selecting all table data require Table Scan, selecting data from multiple tables require parsing of JOIN conditions.

Query Optimizer is to find an efficient execution plan for your query. Even for relatively simple queries, there may be a large number of different ways to access the data to produce the same end result.

The Query Optimizer is a cost-based optimizer. Each execution plan has associated with cost estimation stats so After extracting the information, Query Optimizer chooses the best efficient execution plan and submit a query to query execution engine.

Query Optimizer is working within a time constraint, there’s a chance that the plan selected may be the optimal plan, but it is also likely that it may just be something close to the optimal plan.

Query Execution:

After optimization of the query, the Execution Engine selects resulting plan and retrieve the desired data.

It also stores generated execution plan into the memory for the query caching purpose.
The stored plan cache may reuse for same query execution, which actually saves lots of time of the Query Optimizer.

Before a few days ago, When I was performing query optimization at that time I have seen one warning in the result of SQL Server Query Execution Plan like: “Operator used tempdb to spill data during execution with spill level 2”.
I performed RND on this and came to know very interesting reason this warning.

If you don’t know, please note that Database Statistics are playing main to prepare and generate the Query Execution Plan.

SQL Server Statistics contain different types of meta data information like total number records, indexes, access path and others.

This warning occurred because my database Statistics are not updated.
For example, my table has total 50000 records and Statistics contains only 5000 records information so the planner prepares the query execution plan according to 5000 records only.

While executing the query, If it finds more than 5000 records, It transfers other additional records first into TempDB and continue further process using TempDB. Because It calculated processing capacity only for 5000 records.

This process degrades the overall query performance and that’s why you can see a warning like: “Operator used tempdb to spill data during execution with spill level 2”.

The solution is:

Find Statistics data for tables and indexes and If Statistics data are not updated, update your Statistics data, so that Query Optimizer can prepare the best Query Execution Plan.

Check the Statistics for particular index:
If you find difference between Rows and Rows Sampled, Statistics is not updated.

DBCC SHOW_STATISTICS ("TableName", IndexName);

Update all Statistics for Database:

EXEC sp_updatestats;

Update Statistics for Table:

UPDATE STATISTICS TABLE_NAME;

Update Statistics for Index:

UPDATE STATISTICS TABLE_NAME INDEX_NAME;

In this post, I am sharing two important parameters which might be useful to improve query execution process in MySQL Server.

The Query Optimizer is playing the main role in the Query Execution Process. It is responsible to choose best Query Execution Plan among the list of Plans.
In MySQL, we can control the task of Query Optimizer by setting few parameters.

Why because?

If you have one big SQL Query (included around 40 tables), Planner generates all possible plans mostly It generates 40+ query execution plan so now Query Optimizer will take more time to SELECT best plan.

We should tune this situation by knowing about optimizer_prune_level and optimizer_search_depth parameters.

optimizer_prune_level: Default is ON, It tells Query Optimizer to skip certain plans based on estimated number of rows.

If we have total 40 tables, Planner generates all possible 40+ plans including individual table plan which might not be needed for Query Optimizer.

If you believe that the optimizer missed a better query plan, this option can be switched off (optimizer_prune_level=0) with the risk that query compilation may take much longer.
This variable we can set both GLOBAL and Session level.

SHOW VARIABLES LIKE 'optimizer_prune_level';

SET GLOBAL optimizer_prune_level=0; -- For global change 
SET optimizer_prune_level=0; -- For session specific change

optimizer_search_depth: Default value is 62, the Planner is generating multiple plans, but sometimes It also generates an incomplete plan.

When Query Optimizer starts to scan the plan, we can set optimizer_search_depth value to tell how far each incomplete plan the optimizer should look to evaluate.

If we set a higher value (Max 63), Query Optimizer try to evaluate all incomplete plans which will take more time to execute. If we set lower value, Query Optimizer can skip few incomplete generated plans.

This variable we can set both GLOBAL and Session level.

SHOW VARIABLES LIKE 'optimizer_search_depth';

SET GLOBAL optimizer_search_depth=40; -- For global change 
SET optimizer_search_depth=40; -- For session specific change

The Table Indexes are always the main focus to improve the performance of any SQL Query.
In MySQL, We can provide INDEX Hint to Query Optimizer for choosing our analyzed Index.

In most of the cases, Query Planner and Query Optimizer are accurate to choose perfect Index.
But sometimes, still It is required to give Index Hint for reducing the overall Query Planning Time.

If we know that this Index is good for this query, we should Provide the Hint to Query Optimizer.

Here, You can also access one related article.

MySQL: Script to find Index Usage Statistics of a Database

MySQL supports command like USE INDEX, IGNORE INDEX, FORCE INDEX, which we can use for Index Hint.

The USE INDEX hint tells MySQL to use only one of the named indexes to find rows in the table.
The IGNORE INDEX tells MySQL to not use some particular index or indexes.
The FORCE INDEX hint acts like USE INDEX , with the addition that a table scan is assumed to be very expensive.

Prepared small demonstration on this.

Create a sample table:

CREATE TABLE tbl_EmployeeDetails
(
    EmpID INTEGER PRIMARY KEY 
    ,EmpName VARCHAR(50)
    ,EmpSalary BIGINT
    ,DepartmentName VARCHAR(50)
);

Insert few sample Records:

INSERT INTO tbl_EmployeeDetails
VALUES 
(1,'Anvesh',80000,'Sales')
,(2,'Neevan',90000,'Sales')
,(3,'Jenny',50000,'Production')
,(4,'Roy',60000,'Production')
,(5,'Martin',30000,'Research')
,(6,'Mahi',85000,'Research')
,(7,'Kruti',45000,'Research')
,(8,'Manish',75000,'Research');

Create few sample Indexes:

CREATE INDEX idx_tbl_EmployeeDetails_EmpName ON tbl_EmployeeDetails (EmpName);
CREATE INDEX idx_tbl_EmployeeDetails_DepartmentName ON tbl_EmployeeDetails (DepartmentName);
CREATE INDEX idx_tbl_EmployeeDetails_EmpName_EmpSalary ON tbl_EmployeeDetails (EmpName,EmpSalary);

Using USE INDEX:

SELECT *FROM tbl_EmployeeDetails
USE INDEX (idx_tbl_EmployeeDetails_EmpName)
WHERE EmpName = 'Roy';

SELECT *FROM tbl_EmployeeDetails
USE INDEX (idx_tbl_EmployeeDetails_EmpName_EmpSalary)
WHERE EmpName = 'Roy' AND EmpSalary > 50000;

Using IGNORE INDEX:

SELECT *FROM tbl_EmployeeDetails
IGNORE INDEX (idx_tbl_EmployeeDetails_EmpName)
WHERE EmpName = 'Roy' AND EmpSalary > 50000;

Using FORCE INDEX:

SELECT *FROM tbl_EmployeeDetails
FORCE INDEX (idx_tbl_EmployeeDetails_EmpName)
WHERE EmpName = 'Roy' AND EmpSalary > 50000;

Like any other RDBMS, PostgreSQL generates a Query Execution Plan for each query that it receives.

It is also used Genetic Query Optimizer (GEQO) that does query planning for heuristic search and It reduce query planning time for large and complex queries.

An Optimizer always tries to choose a good Query Execution Plan.

The structure of Query Execution Plan is like a Tree and It contains different information like: type of scan details, join details, sorting details, aggregation details, Estimated rows details, Estimated start-up cost, Estimated total cost and others.

To know about the Query Execution Plan, is very important for all Database Professionals because It helps us to improve the Query Performance.

With the help of Query Execution Plan, We can make our decisions on Index tuning, Join tuning and other.

In PostgreSQL, we have EXPLAIN and EXPLAIN ANALYZE command to know about the Query Execution Plan.

Here, I have used one table tbl_itemtransactions which you can create using below post.

Example of EXPLAIN: It generates the Estimated Query Execution Plan with out executing an actual SQL Query.

EXPLAIN SELECT COUNT(1) FROM tbl_itemtransactions;

/*
Result: 
"Aggregate  (cost=2555525.00..2555525.01 rows=1 width=0)"
"  ->  Seq Scan on tbl_itemtransactions  
(cost=0.00..2216945.00 rows=135432000 width=0)"
*/

Example of EXPLAIN ANALYZE: It first executes the SQL Query and than It generates the actual Query Execution Plan.

EXPLAIN ANALYZE SELECT COUNT(1) FROM tbl_itemtransactions;

/*
Result: 
"Aggregate  (cost=2555525.00..2555525.01 rows=1 width=0) 
(actual time=52660.132..52660.133 rows=1 loops=1)"
"  ->  Seq Scan on tbl_itemtransactions  
(cost=0.00..2216945.00 rows=135432000 width=0) 
(actual time=0.027..46193.484 rows=135432001 loops=1)"
"Planning time: 0.204 ms"
"Execution time: 52660.237 ms"
*/