SQL Server 2016 introduced System-versioned Temporal Table to keep a history of data change by adding additional history table.

In the below article, You can read more about System-versioned Temporal Table and you must know all the rules before creating a System-versioned Temporal Table.

You can also enable System-versioned Temporal Table in the existing table.

Check this demonstration:

Create a table with sample data:

CREATE TABLE tbl_Employees
(
	EmpID INT PRIMARY KEY
	,EmpName VARCHAR(20)
	,EmpDepartment VARCHAR(20)
	,EmpSalary INT
)
GO
 
INSERT INTO tbl_Employees
VALUES 
(1,'Anvesh','Database',90000)
,(2,'Jenny','JAVA',65000)
,(3,'Martin','PHP',85000)
,(4,'Roy','PHP',94000)
GO

Enable System-versioned Temporal Table on this table:

ALTER TABLE tbl_Employees
SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE=dbo.tbl_EmployeesHistory))
GO

You will get a below error because you must configure SYSTEM_TIME column.

Msg 13510, Level 16, State 1, Line 1
Cannot set SYSTEM_VERSIONING to ON when SYSTEM_TIME period is not defined.

Configure SYSTEM_TIME column:

ALTER TABLE tbl_Employees
ADD 
	StartTime DATETIME2 GENERATED ALWAYS AS ROW START DEFAULT GETUTCDATE()
	,EndTime DATETIME2 GENERATED ALWAYS AS ROW END DEFAULT CONVERT(DATETIME2, '9999-12-31 23:59:59.9999999') 
	,PERIOD FOR SYSTEM_TIME (StartTime, EndTime)  
GO

Now, enable System-versioned Temporal Table:

ALTER TABLE tbl_Employees
SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE=dbo.tbl_EmployeesHistory))
GO

SQL Server 2016 introduced a new type of user table that is called System-versioned Temporal Table to keep a full history of data changes.

This is a very good addition to the SQL Server 2016 because to track data change history is always an important task for a big transactional system.

SQL Server already provides few features to track data change like Change Data Capture (CDC), SQL Server Auditing, can create a manual DDL Trigger.

Now, SQL Server 2016 System-versioned temporal table which is something good and very easy to implement on a specific table.
You can audit all data changes and can perform require analysis on it. In Case of invalid delete or update, you can recover your data.

Rules and Requirements:

• Can create your table using SYSTEM_VERSIONING=ON
• Define your history table name HISTORY_TABLE = TableName, which tracks data change history. (It creates automatically).
• Must define two additional columns [ValidFrom] and [ValidTo] with default ROW START and ROW END and must refer to as SYSTEM_TIME period columns.
• Must have a Primary key column.
• Same number of column and structure between both the tables.

Full demonstration:

Create a sampel System Version Table:

CREATE TABLE tbl_Students
(
	StudID INT PRIMARY KEY
	,StudName VARCHAR(50)
	,StudClass CHAR(1)
	,ValidFrom DATETIME2 (2) GENERATED ALWAYS AS ROW START  
	,ValidTo DATETIME2 (2) GENERATED ALWAYS AS ROW END  
	,PERIOD FOR SYSTEM_TIME (ValidFrom, ValidTo)  
) WITH (SYSTEM_VERSIONING = ON (HISTORY_TABLE = dbo.tbl_StudentsHistory));  
GO

Insert few sample records:

INSERT INTO tbl_Students 
(StudID,StudName,StudClass)
VALUES (1,'Roy','A'),(2,'Bony','B'),(3,'Mahi','C')
,(4,'Maria','A'),(5,'Ron','B'),(6,'Murey','C')
GO

Execute few DML statements:

UPDATE tbl_Students SET StudClass = 'B'
WHERE StudID = 4;
GO

DELETE FROM tbl_Students WHERE StudID = 2
GO

Check the result of both tables:

SELECT *FROM tbl_StudentsHistory

StudID  StudName      StudClass ValidFrom                   ValidTo
------- ------------- --------- --------------------------- ---------------------------
4       Maria             A         2017-01-06 05:47:23.73      2017-01-06 05:48:54.43
2       Bony              B         2017-01-06 05:47:23.73      2017-01-06 05:50:12.52


SELECT *FROM tbl_Students

StudID  StudName        StudClass ValidFrom                   ValidTo
------- --------------- --------- --------------------------- ---------------------------
1       Roy                 A         2017-01-06 05:47:23.73      9999-12-31 23:59:59.99
3       Mahi                C         2017-01-06 05:47:23.73      9999-12-31 23:59:59.99
4       Maria               B         2017-01-06 05:48:54.43      9999-12-31 23:59:59.99
5       Ron                 B         2017-01-06 05:47:23.73      9999-12-31 23:59:59.99
6       Murey               C         2017-01-06 05:47:23.73      9999-12-31 23:59:59.99

SQL Server 2016 came with a lot of great features which will help us in our day to day activities.
It introduced a fantastic feature to monitor live query execution using Live Query Statistics option.

We knew about the Actual Execution Plan and Estimated Execution Plan, which gives us the full execution plan of the query.
Now, we have one more option in this category, which is Live Query Statistics.

As per the MSDN,

SQL Server Management Studio provides the ability to view the live execution plan of an active query.
This live query plan provides real-time insights into the query execution process as the controls flow from one query plan operator to another.

We can get overall query progress in real time without needing to wait for the query to complete.
When we are debugging the overall query execution, it helps a lot to debug every block of the query.

How to enable Live Query Statistics option for the SQL Query?

Using SSMS:

In the SQL Editor toolbar, you can find one new button called “Include Live Query Statistics” which is immediately next to “Include Actual Execution Plan” button.

Right click on SQL Query:

Select your query and right click on it. You can select an option like: “Include Live Query Statistics” for that particular query execution.

SQL Server 2016 introduced sys.dm_db_column_store_row_group_physical_stats system view for finding the information on Columnstore row groups and related fragmentation.
This is an extended version of sys.column_store_row_groups.

Using this system view, you can find information like total number of rows physically stored, and a column for the number of rows marked as deleted.

SELECT 
	OBJECT_NAME([i].[object_id]) AS TableName
	,[i].[name] AS IndexName	
	,[csrgps].[state_desc]
	,[csrgps].[total_rows]
	,[csrgps].[deleted_rows]
	,[csrgps].[size_in_bytes]
	,[csrgps].[trim_reason_desc]
	,[csrgps].[transition_to_compressed_state_desc]
	,[csrgps].[has_vertipaq_optimization]	
	,[csrgps].[created_time]
	,[csrgps].[closed_time]
	,100 * (ISNULL(csrgps.[deleted_rows],0)) / total_rows AS 'IsFragmentation'
FROM sys.indexes AS i
INNER JOIN sys.dm_db_column_store_row_group_physical_stats AS csrgps
	ON [i].[object_id] = [csrgps].[object_id] 
		AND [i].index_id = [csrgps].index_id  
ORDER BY [TableName], [IndexName] 
GO

SQL Server 2016 introduced sys.dm_column_store_object_pool system view for finding the memory pool usage by different types of Columnstore Index objects.

I am sharing one T-SQL Script to find memory pool usage by Columnstore Indexes.
You can find one object_type column in the result set of below query.

The information of object_type column is:

1 = COLUMN_SEGMENT
2 = COLUMN_SEGMENT_PRIMARY_DICTIONARY
3 = COLUMN_SEGMENT_SECONDARY_DICTIONARY
4 = COLUMN_SEGMENT_BULKINSERT_DICTIONARY
5 = COLUMN_SEGMENT_DELETE_BITMAP

SELECT 
	OBJECT_NAME(csop.[object_id]) AS TableName
	,[i].[name] AS IndexName
	,[c].[name] AS ColumnName
	,[csop].[column_id]
	,[csop].[row_group_id]
	,[csop].[object_type_desc]
	,[csop].[access_count]
	,[csop].[memory_used_in_bytes]
	,[csop].[object_load_time]
FROM sys.dm_column_store_object_pool AS csop
LEFT OUTER JOIN sys.index_columns AS ic
	ON [ic].[index_column_id] = [csop].[column_id] 
		AND [ic].[index_id] = [csop].[index_id] 
		AND [ic].[object_id] = [csop].[object_id]
LEFT OUTER JOIN sys.columns AS c
	ON [ic].[object_id] = [c].[object_id] 
		AND [ic].[column_id] = [c].[column_id]
LEFT OUTER JOIN sys.indexes AS i
	ON [i].[object_id] = [csop].[object_id] 
		AND [i].[index_id] = [csop].[index_id]
ORDER BY [csop].[memory_used_in_bytes] DESC
GO

Microsoft SQL Server 2016 comes with a variety of new additions and I am happy for exploring the new features of SQL Server 2016.

Microsoft SQL Server 2014 introduced Hekaton engine which used for In-Memory Database objects.

SQL Server 2014: Create Memory Optimized File Group and Table

SQL Server 2016 introduced Columnstore index for Memory-Optimized Table which was not possible previously.
If you are planning to store large amount of data In Memory, you can apply Columnstore Index on that In Memory OLTP Table.

It comes with one condition: you can not apply Columnstore Index on the particular column. It must apply to all the columns of In-Memory Table.

Small demonstration with all the possible errors:

Create a sample memory optimized table with column store index:

CREATE TABLE tbl_Employees
(
	EmpID INT
	,EmpName VARCHAR(50)
	,EmpSalary INT
	,EmpDOJ DATETIME
	,INDEX idx_tbl_Employees_columnstore CLUSTERED COLUMNSTORE 
)
WITH (MEMORY_OPTIMIZED = ON)
GO

You will get a below error:

Msg 41321, Level 16, State 7, Line 1
The memory optimized table 'tbl_Employees' with DURABILITY=SCHEMA_AND_DATA must have a primary key.

Add primary key on script and execute it:

CREATE TABLE tbl_Employees
(
	EmpID INT PRIMARY KEY
	,EmpName VARCHAR(50)
	,EmpSalary INT
	,EmpDOJ DATETIME
	,INDEX idx_tbl_Employees_columnstore CLUSTERED COLUMNSTORE 
)
WITH (MEMORY_OPTIMIZED = ON)
GO

You will get a below error:

Msg 12317, Level 16, State 78, Line 3
Clustered indexes, which are the default for primary keys, are not supported with memory optimized tables. Specify a NONCLUSTERED index instead.

Apply Non Clustered Primary key and execute it:

CREATE TABLE tbl_Employees
(
	EmpID INT PRIMARY KEY NONCLUSTERED
	,EmpName VARCHAR(50)
	,EmpSalary INT
	,EmpDOJ DATETIME
	,INDEX idx_tbl_Employees_columnstore CLUSTERED COLUMNSTORE 
)
WITH (MEMORY_OPTIMIZED = ON)
GO

If you have not created Memory Optimized File group and related files, you may get a below error:

Msg 41337, Level 16, State 100, Line 1
Cannot create memory optimized tables. To create memory optimized tables, the database must have a MEMORY_OPTIMIZED_FILEGROUP that is online and has at least one container.

Create memory optimized file group and data files:

USE [master]
GO
ALTER DATABASE [Database_Name] 
ADD FILEGROUP [Memory_Optimized] CONTAINS MEMORY_OPTIMIZED_DATA 
GO
ALTER DATABASE [Database_Name] 
ADD FILE ( NAME = N'HekatonMemory', FILENAME = N'C:\Hekaton\hek.mdf' ) 
TO FILEGROUP [Memory_Optimized]
GO

SQL Server 2016 introduced one new DMV is sys.dm_exec_function_stats which returns the aggregate performance statistics for cached functions.

It is very similar to sys.dm_exec_query_stats and it returns information about scalar functions, In-Memory functions and CLR functions.

You can also get the number of execution count of functions which helps you to troubleshoot performance related problems.

SELECT TOP 30
	d.object_id
	,d.database_id
	,OBJECT_NAME(object_id, database_id) AS FunctionName
	,d.cached_time
	,d.last_execution_time
	,d.total_elapsed_time
	,d.total_elapsed_time/d.execution_count AS [avg_elapsed_time]
	,d.last_elapsed_time, d.execution_count  
FROM sys.dm_exec_function_stats AS d  
ORDER BY [total_worker_time] DESC;

SQL Server 2014 introduced DBCC CLONEDATABASE for creating a blank database with only Schema and Statistics.

SQL Server 2016 added support of the Query Store in DBCC CLONEDATABASE.

This feature is very helpful in certain conditions.
For example, Your production database size is around 800 GB, and You faced some performance related issues because the problem of outdated Statistics or Indexes issue.

Now the situation is, you can do a test and trial in your Production Database, so DBA takes a copy of Production Database and restore it in different server for further investigation.

Do you think, this is the only one solution because Its require additional time and infrastructure.

In this kind of situation, you can use the DBCC CLONEDATABASE feature of SQL Server 2014.

It creates a blank database copy with the only Schema, Statistics and Query Store information. It is not copying the data, so it does not require any additional space.

You can find the same statistical information in the new clone database which generates the same execution plan for your investigation.

A Query Execution Plan is prepared to base on Database Statistics so If Database Statistics are same then you check your query execution plan without data.

Once you are finished with your task with Clone Database, you can delete it.

How internally SQL Server creates a Clone Database?

It validates the source database and acquires the shared lock on the source database

It creates a snapshot of the source database (Include only: Schema, Statistics, Query Store)

Create Clone Database and acquires an exclusive lock on it

Copy the metadata, schema, statistics, query store information from the source database to clone database

Release all locks from Clone Database

How to create a Clone Database?

Use DBCC CLONEDATABASE command to create a Clone Database.

DBCC CLONEDATABASE (MyDatabase,MyDatabase_Clone);

SQL Server 2016 introduced the COMPRESS() and DECOMPRESS() function which we can use to compress column data of a table.

The Compression of the column is not a good always and not a good for all the column. For that, there are different ways for compression.

But we can use COMPRESS() on those columns which size are very big and which are not most in use.
The reason is, compressed column requires to be decompressed using DECOMPRESS() while selecting the data which will reduce query performance.

If we are facing a size related problem, we can think for COMPRESS() on those big size columns which are not most in use.

Please check the below demonstration on COMPRESS() and DECOMPRESS():

Create a sample table:

CREATE TABLE tbl_DumpData
(
	ID INT 
	,RandomData NVARCHAR(MAX)
	,CONSTRAINT pk_tbl_DumpData_ID PRIMARY KEY(ID)
)
GO

Insert dummy data:

;WITH CTE AS
(
	SELECT 1 ID
	UNION all
	SELECT ID + 1
	FROM CTE   
	WHERE ID + 1 <= 1000000
)
INSERT INTO tbl_DumpData(ID,RandomData)
SELECT 
	ID
	,CONCAT(CAST(NEWID() AS NVARCHAR(MAX)), ' - I am follower of dbrnd.com') AS RandomData
FROM CTE 
OPTION (MAXRECURSION 0)
GO

Create a sample table to store Compressed Data:

CREATE TABLE tbl_DumpData_Compressed
(
	ID INT 
	,RandomData NVARCHAR(MAX)
	,CONSTRAINT pk_tbl_DumpData_Compressed_ID PRIMARY KEY(ID)
)
GO

Insert Compressed data from the original table:

INSERT INTO tbl_DumpData_Compressed
SELECT ID, COMPRESS (RandomData) AS RandomData
FROM tbl_DumpData

Compare the size of both table:
You will get to know that, compressed table size is smaller than original table

SP_Spaceused 'tbl_DumpData'
GO
SP_Spaceused 'tbl_DumpData_Compressed'
GO

Now, do DECOMPRESS and select your data:

SELECT
	ID
	,CAST(DECOMPRESS(RandomData) AS NVARCHAR(MAX)) AS RandomData 
FROM tbl_DumpData_Compressed

To complete this article, you must visit my yesterday’s post where I explained about the use of sp_estimate_data_compression_savings system stored procedure of SQL Server.

In this post, I continue my demonstration and am using the same example of a table from the previous post. SQL Server 2016 introduced the Data Compression algorithm for compressing or reducing the size of an object.

For more detail – visit this official LINK.

In this previous post, We can see 40% space that we can reduce of a tbl_Dumpdata table.

How?

I created a clustered index on that table, so for data compression, we have to REBUILD that index with option DATA_COMPRESSION.

Rebuild Index with DATA_COMPRESSION:

ALTER INDEX pk_tbl_DumpData_ID 
ON dbo.tbl_DumpData
REBUILD PARTITION = ALL 
WITH (DATA_COMPRESSION = PAGE)
GO

Now, check the space for PAGE option:
Check the below result, you can find the same value of column size_with_current_compression_setting and size_with_requested_compression_setting.

EXEC sp_estimate_data_compression_savings 
    @schema_name = 'dbo', 
    @object_name = 'tbl_DumpData', 
    @index_id = NULL, 
    @partition_number = NULL, 
    @data_compression = 'PAGE'