In the previous two posts, I shared basic about Forwarding Records of Heap Table and shared a script to find the count of Forwarded Records in SQL Server.

In this post, I am sharing a practical demonstration on Forwarding Records like generate Forwarding Records and then Remove those Forwarding Records.

Please check the below full demonstration:

Create a sample HEAP table:

CREATE TABLE tbl_Heap
(
	Col1 INT IDENTITY(1,1)
	,Col2 VARCHAR(1000)	
)
GO

Insert sample records with VARCHAR(5) size:

INSERT INTO tbl_Heap (Col2) VALUES
('dbrnd')
GO 1000

Now, Update the data from VARCHAR(5) to VARCHAR(31):

UPDATE tbl_Heap
SET Col2 = 'Database Research & Development'
GO

Check the count of Forwarding Records:

For removing Forwarding Records, we should REBUILD the table:

ALTER TABLE tbl_Heap REBUILD
GO

Now, Check the new count of Forwarding Records:
The count is 0 now.

In the previous post, I shared basic about Forwarded Records in Heap table of SQL Server.
In this post, I am sharing a script to check the count of Forwarded Records of a Table.

In the below script, you can add a filter of your database and table for getting the number of Forwarding Records.

SELECT
	OBJECT_NAME(object_id) as TableName
	,index_type_desc
	,page_count
	,avg_page_space_used_in_percent
	,avg_record_size_in_bytes
	,forwarded_record_count
FROM sys.dm_db_index_physical_stats
(
	DB_ID('database_name')
	,OBJECT_ID('table_name')
	,NULL
	,NULL
	,'DETAILED'
)
GO

In this post, I am sharing an important note on Forwarded Records in the Heap table of SQL Server. Whenever DBA or Developer is facing performance issue, they are always talking about the table or index fragmentation, but no-one is discussing the Forwarded Records.

What is heap Table

Simple Example to understand Forwarded Records:

You create a heap table, and one of the column types is VARCHAR(200).
You inserted 100000 records and added a small size of the string in VARCHAR(200) like “dbrnd” value which is only VARCHAR(5).

Now next, you require updating the value from “dbrnd” to “Database Research & Development” which is now VARCHAR(31).
Previously, we stored our data based on VARCHAR(5), but while updating we need VARCHAR(31).

Before the update, all data correctly save in data page and for new update no more free space available on the data page. So SQL Server internally moves new data into the new page and assigned a new Relative Identifier (RID).

Now old page contains forwarding pointer to new RID and new records also back pointer to old records.

– Called as Forwarded Records and which is also one type of internal fragmentation.

The biggest drawback of Forwarded Record is, it requires high I/O because SQL Server has to jump between the old data page and new data page for getting data.

Few more following topics, which I will publish from tomorrow:
How to find the Forwarded Records?
How to remove Forwarded Records?

 Many people asked me, Are you doing an only blogging or consulting? Every time my clear answer was, I am doing full-time Job as DB Lead in one of the leading IT Companies (working for 9 hours to 10 hours) and

As you guys know, that I have shared loads of SQL Server Interview questions and answers.
Let me tell you a story behind this,

Whenever I free in a weekend; I always try to attend walk-in interviews which have been arranged by most of the IT Companies so that I can find some real interview questions and can share with you guys.

Wait wait wait…. I know most of the people are thinking that I am running for a job… which is false.

I am not going to change my current company, but I am attending most of the interviews because I want to measure myself so that can share something more and can also take superb interviews that I am also doing in my current company. Additionally spending minimum 3 hours to 4 hours every day for blogging…

I know, you guys are suffering my blah blah blah…. from the above two paragraphs, but for a golden question, you guys could suffer sometimes.

Last week, I attended a walk-in interview for DBA position at Tech Mahindra Hyderabad, India.
They asked few questions on Clustered Index, and I was empty entirely.

Questions were:

• Can we disable the Clustered Index?
• Have you ever tried to disable Clustered Index?
• Once you disable the Clustered Index, Can we access the table?
• Your table has Clustered Index + NonClustered Indexes. If you turn off the Clustered Index, Can you access your table using NonClustered Indexes?
• Can we take the table in offline mode by disabling the Clustered Index?
• Ok, If you disable the Clustered Index, how can you enable it again?

My simple answers were:

• I don’t know it is possible or not
• I never tried to disable the Primary key, and if it is possible
• I don’t know how we can find the data path to access the table

Now, I don’t want to write more blah, blah, blah… Find the below full demonstration and test it yourself and at the end, you will get all answers to above questions.

All the best....

Now, one more question to you guys. If you have heap table, what ?, Can anyone share their thoughts via putting some comments?

Create a sample table:

CREATE TABLE tbl_Students 
(
	StudID INT IDENTITY(1,1)
	,Name VARCHAR(10)
	,CONSTRAINT pk_tbl_Students_StudID PRIMARY KEY(StudID)
)
GO

Create a NonClustered index on Name column:

CREATE NONCLUSTERED INDEX idx_tbl_Students_Name ON tbl_Students(Name)
GO

Insert 10000 dummy records:

INSERT INTO tbl_Students(Name) VALUES ('Anvesh')
GO 10000

Now, Disable the Clustered Index:
It also disables the all Nonclustered Indexes of a table. You can check below warning message***

ALTER INDEX pk_tbl_Students_StudID ON dbo.tbl_Students DISABLE
GO

Warning: Index 'idx_tbl_Students_Name' on table 'tbl_Students' was disabled as a result of disabling the clustered index on the table.

Try to select a table:
You will get a below error, You cannot select the table data because of query optimizer unable to find the access path of data.

SELECT *FROM tbl_Students
GO

Msg 8655, Level 16, State 1, Line 1
The query processor is unable to produce a plan because the index 'pk_tbl_Students_StudID' on table or view 'tbl_Students' is disabled.

The solution is, Rebuild your Clustered Index and you will get your table back:

ALTER INDEX pk_tbl_Students_StudID ON dbo.tbl_Students REBUILD
GO

SQL Server: Script to find Heap Table without having an Index

SQL Server: Script to find Index Average Fragmentation in Percentage

SQL Server: Script to find Missing Indexes

SQL Server: Script to find usage of Database Indexes

SQL Server: Script to find Index Operational Stats

What is your best practice to perform large-scale Index operations?

Make sure the transaction log has been backed up and truncated before running large-scale index operations online, and that the log has sufficient space to store the projected index and user transactions.

Consider setting the SORT_IN_TEMPDB option to ON for the index operation. This separates the index transactions from the concurrent user transactions. The index transactions will be stored in the tempdb transaction log, and the concurrent user transactions will be stored in the transaction log of the user database.

This allows for the transaction log of the user database to be truncated during the index operation if it is required.

What is Online Indexing?

The ONLINE option allows concurrent user access to the underlying table or clustered index data and any associated nonclustered indexes during these index operations.

DBAs can create, rebuild, or drop indexes online. The index operations on the underlying table can be performed concurrently with update or query operations.

The Online indexing feature is very helpful for environments that run 24 hours a day, seven days a week. The Online Indexing feature is available only in the Enterprise Editions.

Can we create index on table variables?

Creating an index on a table variable can be done implicitly within the declaration of the table variable by defining a primary key and creating unique constraints. The primary key will represent a clustered index, while the unique constraint a non clustered index.

What are your best practices to create an Index?

• Understand the characteristics of the database (OLTP / OLAP).
• Keep indexes lean.
• Understand the characteristics of the most frequently used columns and queries.
• Choose the right index type.
• Create the clustered index on every table.
• Keep the length of the index key short for clustered indexes.
• Columns that are of the ntext, text, image, varchar(max), nvarchar(max), and varbinary(max) data types cannot be specified as index key columns.
• Consider using filtered indexes on columns that have well-defined subsets.
• Be sure to eliminate duplicate indexes, that is, multiple indexes created on the same set of columns.
• Consider the order of the columns if the index will contain multiple columns.
• Determine the Fill Factor.

How more indexes makes your INSERT slower?

The number of indexes on a table is the most dominant factor for insert performance. The more indexes a table has, the slower the execution becomes. The insert statement is the only operation that cannot directly benefit from indexing because it has no where clause.

Adding a new row to a table involves several steps. First, the database must find a place to store the row.

If there are indexes on the table, the database must make sure the new entry is also found via these indexes. For this reason it has to add the new entry to each and every index on that table. The number of indexes is therefore a multiplier for the cost of an insert statement.

Moreover, adding an entry to an index is much more expensive than inserting one into a heap structure because the database has to keep the index order and tree balance. That means the new entry cannot be written to any block—it belongs to a specific leaf node.

Once the correct leaf node has been identified, the database confirms that there is enough free space left in this node. If not, the database splits the leaf node and distributes the entries between the old and a new node.

Generally, when you should perform your index tunning exercise?

Index tuning isn’t something you do once a year. It’s something that you need to do iteratively– that means every month. Over time, data sizes change, user activity changes, and the SQL Server optimizer changes. Each of these things mean that indexes that are best for an application will also change. As you tune indexes, your query plans will change and you’re very likely to see more opportunities to add, drop, and combine indexes emerge. Because of this, you want to do a few changes every month.

Has default Index created on foreign key columns?

When a foreign key constraint is defined for a table, an index is not created by default by SQL Server.

However, it’s not uncommon for developers and database administrators to add them manually.

Indexes on foreign keys can provide performance benefits for table joins involving the primary and foreign key, and they can impact performance when the primary key value is updated, or if the row is deleted.

In which columns, you should consider your indexes?

Indexes should be considered on all columns that are frequently accessed by the WHERE, ORDER BY, GROUP BY, TOP, and DISTINCT clauses. Without an index, each of these operations will require a table scan of your table, potentially hurting performance.

When index needs change and you want to test how removing an index will affect performance, you have two options – you can disable or drop the index. Which is better?

DISABLING AN INDEX is better than to drop an Index.

When we Disable an Index, The metadata about the index is retained, as are the usage statistics in sys.dm_db_index_usage_stats.

The index is not usable by the query optimizer.

Disabling a nonclustered index will deallocate the index pages – the space is freed in the database.

Disabling a clustered index has additional effects. The data in the table still exists, but will be inaccessible for anything other than a drop or rebuild operation.

T-SQL Script to DISABLE an INDEX?

ALTER INDEX IX_IndexName ON Schema.TableName DISABLE;

How you can re-enable the disable index?

If you want to re-enable the index, you need to rebuild it.

ALTER INDEX IX_IndexName ON Schema.TableName REBUILD;

As a DBA, share your checklists and best practice for performance tuning of Indexes.

• Periodically, run the Database Engine Tuning Advisor against current Profiler traces to identify potentially missing indexes.
• Remove indexes that are never used.
• Don’t accidentally create redundant indexes.
• As a rule of thumb, every table should have at least a clustered index. Generally, but not always, the clustered index should be on a column that monotonically increases — such as an identity column, or some other column where the value is increasing — and is unique. In many cases, the primary key is the ideal column for a clustered index.
• Since you can only create one clustered index per table, take extra time to carefully consider how it will be used. Consider the type of queries that will be used against the table, and make an educated guess as to which query (the most common one run against the table, perhaps) is the most critical, and if this query will benefit most from having a clustered index.
• If a column in a table is not at least 95% unique, then most likely the query optimizer will not use a non-clustered index based on that column. Because of this, you generally don’t want to add non-clustered indexes to columns that aren’t at least 95% unique.
• Keep the “width” of your indexes as narrow as possible. This reduces the size of the index and reduces the number of disk I/O reads required to read the index, boosting performance.
• If possible, avoid adding a clustered index to a GUID column (uniqueidentifier data type).
• Indexes should be considered on all columns that are frequently accessed by the JOIN, WHERE, ORDER BY, GROUP BY, TOP, and DISTINCT clauses.
• Don’t automatically add indexes on a table because it seems like the right thing to do. Only add indexes if you know that they will be used by queries run against the table.
• When creating indexes, try to make them unique indexes if at all possible. SQL Server can often search through a unique index faster than a non-unique index because in a unique index, each row is unique, and once the needed record is found, SQL Server doesn’t have to look any further.
• If you perform regular joins between two or more tables in your queries, performance will be optimized if each of the joined columns have appropriate indexes.
• Don’t automatically accept the default value of 100 for the fill factor for your indexes. It may or may not best meet your needs. A high fill factor is good for seldom changed data, but highly modified data needs a lower fill factor to reduce page splitting.
• Don’t over index your OLTP tables, as every index you add increases the time it takes to perform INSERTS, UPDATES, and DELETES. There is a fine line between having the ideal number of indexes (for SELECTs) and the ideal number to minimize the overhead that occurs with indexes during data modifications.

In this post, I am sharing a script for a Database Administrator to find SQL Server Heap Tables which are not having a clustered index.

A Table without a clustered index is called as Heap Table and Data is stored in the Heap without specifying an order.

I have also published one article on, What is Heap Table?
You can visit this article here:

Sometimes, database developer forgets to apply cluster index on a particular table and after that, the same table may have a lot of data which affect the query performance without having a clustered index.

Below is a script:

SELECT 
	OBJECT_NAME(object_id) AS TableName
	,type_desc AS TableType
	,allow_row_locks
	,allow_page_locks
FROM sys.indexes 
WHERE index_id = 0

In the previous post, I had explained the basic theory on the Heap Table of Database Technology.
In this post, I am going to explain the Heap table terminology of MySQL.

MySQL Memory Storage Engine also called as Heap Storage structure.

Using Memory Storage Engine, you can store a table of data in server memory for fast retrieval.
This table of data also supports B-Tree index so you can get a result in a best-optimized way. Auto_Increment also supported by the Tables of Memory Engine.

There might be chances of loss data when any hardware crashes or power outage of Server Memory.

You can store and use only those data which are in temporary use and make sure that data in memory engine has always back up in physical stored table.

When to use Memory Engine of MySQL ?

If data are non-critical, need fast retrieval and only for read-only access with a minimum of updates.
The volume of data can be fit into Memory area because it internally uses the swap out of virtual memory pages.

Brief details of Memory Engine Mechanism:

The table of Memory Engine stores it’s table definition into one disk file, and extension of this file is .frm.
All other data stores in a specified area of memory.
The data in memory table is not only for one single connection, it is available for all running related connections and sessions of MySQL Server.

Create sample table using Memory Engine:

CREATE TABLE Test (TestNo INT,TestName VARCHAR(50)) ENGINE = MEMORY;

This all basic about MySQL Memory or Heap Engine.

Any confusion, like difference between Temporary table and Memory table.
If yes, then please visit next post.

You can find an N number of articles on Heap Tables. But still, I would like to write a theory on Heap Table.

In simple term, Table without a clustered index is called as Heap Table. Data stored in a heap without specifying an order.

Initially, data stored in the order of its inserted, but later database engine can move data around in the heap so you cannot predict the order of data, and because of this you have to write an ORDER BY clause for each and every use.

When you can use Heap Structure?

Heap table is not advisable to use, but still, sometimes database architectures are using the Heap table.

You can use Heap structure when the table is tiny. If a table has a slight amount of data, then you can use a Heap structure because in that condition Table Scan performs faster than Index Scan.

Heap Structure use RID (Row Identifier) to fetch data from file and data page. RID contains file number, data page number and slot details of the page.

When you should not use heap table?

Simple, when the amount of data is higher and require a predefined ORDER BY to the SELECT bulk of data.

You cannot use a heap table when queries are frequently executing.
You cannot use a heap table when data are grouped together because a group of the data is required proper ordered of data.