In this post, I am sharing an important tip for PostgreSQL DBAs.

I recently started work with a new PostgreSQL Client. They have total 326 databases in their PostgreSQL Server. They are taking full backup for all the database and this process is taking around total 7 hours.

When I looked this, my first step was to find database statistics. I have found 25% of databases are not being updated frequently.

I added one additional check in a backup process like “skip the backup for those databases which have not been changed for any operations (DDL or DML operation)”.
After this check, the backup process is taking around total 5 hours to complete the job.

Using pg_stat_database, you can check the stats of database and can compare the value for tuple columns with the old value. If you found any changes, your database has been changed.

If any of your DBA reset the PostgreSQL database statistics using something like pg_stat_reset, you have to add some more extra logic to understand the difference between statistics.

Below is a sample demonstration:
Create a backup table to store DatabaseStats:

CREATE TABLE public.tbl_DatabaseStats AS
SELECT 
	datname
	,tup_inserted
	,tup_updated
	,tup_deleted	
FROM pg_stat_database;

Perform few DDLs and DMLs actions:

CREATE TABLE public.tbl_Students(rno int, name character varying);
INSERT INTO public.tbl_Students VALUES(1,'Anvesh');
CREATE TABLE public.tbl_Test88(id int);

Compare the result of below two queries:
If you find differences on any of the columns of your Database, your database stats has been changed and you should take the backup of your database.

SELECT 
	datname
	,tup_inserted
	,tup_updated
	,tup_deleted	
FROM pg_stat_database;

SELECT *FROM public.tbl_DatabaseStats;

In this post, I am sharing one T-SQL Script to take full backup for all the databases of SQL Server.

There are different types of backups and techniques available in the SQL Server, but I always believe in DBA Scripts which you can modify as per your requirements.

Apart from scheduled backups, sometimes DBA requires ad-hoc full backups of databases which they can do using this script.

Script to take full backup of databases:
You can change the path name (‘C:\TempBackup\’) and replace with your path name.

DECLARE @dbName	NVARCHAR (255); 
DECLARE @SQL NVARCHAR(4000); 
 
DECLARE DBBackupCur 
CURSOR FOR 
    SELECT name FROM sys.databases WITH (NOLOCK) 
    WHERE name NOT IN ('master','model','msdb','tempdb')  
    OPTION (RECOMPILE); 
 
OPEN DBBackupCur;     
FETCH NEXT FROM DBBackupCur INTO @dbName 
WHILE (@@fetch_status <> -1) 
BEGIN 
    IF (@@fetch_status <> -2) 
    BEGIN 
        SET @SQL = N'BACKUP DATABASE [' + @dbName + N'] TO DISK = ''C:\TempBackup\' + @dbName + N'.bak'''; 
        EXECUTE sp_executesql @SQL 
        PRINT N'Backup completed: ' + @dbName 
    END; 
    FETCH NEXT FROM DBBackupCur INTO @dbName; 
END; 
CLOSE DBBackupCur; 
DEALLOCATE DBBackupCur; 
GO

Greenplum is based on MPP (massively parallel processing) architecture. It built on shared nothing mode parallel processing in which all child segments own separate resources.

Database Theory: What is Parallel Query Processing (Parallel Database System)?

When you are taking database backup in Greenplum, it executes a backup process in different stages and executes in parallel.

For GP DBA, this is very important for knowing that in which machine or segment backup process is running. They may have few other questions like Is backup process completed for a master segment?, If it is completed for a master segment, in which child segment it is running currently.

You can access this article for troubleshooting Greenplum running database backup process.

You can start Greenplum full database backup using below command:

gpcrondump -x database_name -u /home/gpadmin/dbrnd_backup/

It first takes a backup of the master node (all metadata or pg_catalog), while during this time you cannot perform any DDL or any other operation on Greenplum database server like CREATE TABLE, DROP TABLE, TRUNCATE TABLE.
It takes only a few minutes to complete the backup of a master node.

How can you check that, Is Greenplum database backup process running on master node?

The first step is to check if the message “Starting Dump process” is on the screen
After this message, you can find a subsequent message “Releasing pg_class lock” that means backup process completed for master segment (meta data backup).

You can measure the time difference between “Starting Dump process” and “Releasing pg_class lock” that is the total time taken for master segment backup.

If you are unable to find “Releasing pg_class lock” on the screen and find only “Starting Dump process” for a long time, there is 99% problem with a master segment and problem like database lock, network issue, disk failure.

How can we check that, Is Greenplum database backup process running on child nodes?

After taken a master node back up, a gp_dump command will be sent to all the segments to start a copy of the data using COPY command.

We can check the running processes in child nodes and can find COPY command and db_dumps directory in the process list. You can get the list of Linux running processes using a command: ps -ef.
Below is a sample process message:

postgres: port 40002, gpadmin database_name [local] con808 [local] cmd880 COPY

Important Note:

Please do not kill long running Greenplum backup from only master node because it may continue the backup process on child segments. Your one wrong kill command can fail your whole Greenplum cluster.

If you guys know how to kill long running backup in Greenplum, please share full detail with me because I am also looking for the solution for killing backup process at all child segments.

In this post, I am sharing different types of options to copy your MySQL Database from one server to another server.

The migration of data, objects or database is a ubiquitous task for any DBA so as a blogger I can’t skip this kind of topics.

Common and General option:

Server 1: Take the backup in .sql file
– -databases option adds the CREATE DATABASE in dump file, so that you can restore same backup in any other server without creating a blank database.

my-shell> mysqldump --databases db_name > mydump.sql

Next, copy mydump.sql file from Server 1 to Server 2.

Server 2: Restore the backup

my-shell> mysql < mydump.sql

My favourite option:

Use pipe and copy the database from one server to another, without taking or moving the physical file.

mysqldump -uuser -ppassword db_name | mysql -hremoteserver -uremoteuser -premoteserverpassword

For all database:

mysqldump --all-databases -uuser -ppassword | mysql -hremoteserver -uremoteuser -premoteserverpassword

I know this is not a new topic, but many SQL Database Administrator doesn’t know about the importance of LSNs, so I am sharing one of the important discussion about the SQL Server LSNs which are very necessary for Database backup and restore activities.

What is LSN?

Every record in the SQL Server transaction log is uniquely identified by a Log Sequence Number (LSN).
The LSNs are sequential, and in the particular ordered, new LSN is always higher than the old LSN.
The LSN is a type of Numeric(25,0), so we can compare different LSNs using equality operators.

The importance of LSN:

Generally, when we are restoring database, we are restoring in this sequence: Full Database backup -> Differential backup -> Transaction Log backup.

We cannot restore the Differential backup before the Full database backup because LSN is sequentially starting from the full database backup.

Before restoring of the Differential backup and Transaction Log backup, it checks FirstLSN, LastLSN, CheckpointLSN, DatabaseBackupLSN columns.

If LastLSN of the first file and FirstLSN of the second file doesn’t match, it fails to restore Differential backup and Transaction log backup.

The sequence of LSNs is critical to restoring your database because all different database backup files virtually connected with each other.

The first full database backup will always have a DatabaseBackupLSN of zero and FirstLSN is same as CheckpointLSN.

The Full database backup CheckpointLSN is a
DatabaseBackupLSN for all differential databases.

The LastLSN of Differential backup will be between FirstLSN and LastLSN of the Transaction Log backup.

I have published my first guest post on www.sqlmvp.org.
I have written about SQL Server Strip Database Backup and Restore for the large size of the Database.

Here, You can visit this article.

SQL Server: Have a large Database, use Striped Database Backup & Restore

In our organization, junior database developers are learning PostgreSQL, and they require a sample database from our Validation Database Server.

In this scenario, Database Administrator takes PostgreSQL backup using pg_dump utility and restore that dump into the development environment.

A pg_dump is a PostgreSQL utility which is used for PostgreSQL database backup and restore.

Another option is, directly copy the database from one server to another server using pg_dump + psql. In this scenario, Database Administrator does not require any intermediate database backup file.

Open your windows command prompt and go to \PostgreSQL\bin directory where you can execute and test below two different commands.

Copy database from one server to another server (using, pg_dump | psql):

pg_dump -h host1 dbname | psql -h host2 dbname

Copy database, Using pg_dump utility for backup and restore:

Take full database backup:

pg_dump -U username database_name > filepath\backup.sql

Restore database backup:

pg_dump -U username database_name < filepath\backup.sql

PostgreSQL: File System Level Backup to just Copy DATA Directory

In this post, I am sharing a T-SQL script to find all Backups related history information of SQL Server.

A Database Administrator can use this script for different types of monitoring.

This script prepared such a way that, you guys can also find information about LSN (Log Sequence Number) information which helps you to identify broken chain.

I was facing a very common issue like Database out of Sync in Log Shipping Setup. I used this script to find out backup related all history and their LSN information.

Using this script, you can find different types of information like Backup media, Type of Backup, Taken Duration.

T-SQL Script, It returns all Backups history information:

SELECT 
	bs.database_name AS DatabaseName
	,CASE bs.type
		WHEN 'D' THEN 'Full'
		WHEN 'I' THEN 'Differential'
		WHEN 'L' THEN 'Transaction Log'
	END AS BackupType
	,CAST(DATEDIFF
		(SECOND,bs.backup_start_date,bs.backup_finish_date) 
		AS VARCHAR(4)) + ' ' + 'Seconds' AS TotalTimeTaken
	,bs.backup_start_date AS BackupStartDate
	,CAST(bs.first_lsn AS VARCHAR(50)) AS FirstLSN
	,CAST(bs.last_lsn AS VARCHAR(50)) AS LastLSN
	,bmf.physical_device_name AS PhysicalDeviceName
	,CAST(CAST(bs.backup_size / 1000000 AS INT) AS VARCHAR(14)) 
		+ ' ' + 'MB' AS BackupSize
	,bs.server_name AS ServerName
	,bs.recovery_model AS RecoveryModel
FROM msdb.dbo.backupset AS bs
INNER JOIN msdb.dbo.backupmediafamily AS bmf
	ON bs.media_set_id = bmf.media_set_id
WHERE bs.database_name = 'Your_Database_Name'
ORDER BY 
	backup_start_date DESC
	,backup_finish_date

In this post, I am going share scripts to take all database and log backups using SQL Server PowerShell.

Using T-SQL, we can also take all databases backup using one single script, but some of the Database Administrators like to perform this kind of task using PowerShell because sometimes it is much faster and it is executing from outside of the SQL Server.

SQL Server 2012 cmdlet introduced a new command to take database backups and the command name is “Backup-SqlDatabase”.

Execute below command in PowerShell to get the help of “Backup-SqlDatabase”:

help Backup-SqlDatabase -full

PowerShell script to take all database backups:
Generating file name with the date.

$BackupDate = Get-Date -Format MMddyy
foreach($database in (Get-ChildItem)) { 
$dbName = $database.Name
$bakFilePath = "C:\temp\" + $dbName + "_" + $BackupDate + ".bak"
Backup-SqlDatabase -Database $dbName -BackupFile $bakFilePath -Initialize }

PowerShell script to take all database log backups:
Generating file name with the date.

$BackupDate = Get-Date -Format MMddyy_hhmm
foreach($database in (Get-ChildItem)) {
$dbName = $database.Name
$trnFilePath = "C:\temp\" + $dbName + "_" + $BackupDate + ".trn"
Backup-SqlDatabase -Database $dbName -BackupAction Log -BackupFile $trnFilePath -Initialize}

Before few days back, One of the SQL Server Developer asked me this question.
How to generate full database schema script or data script in PostgreSQL?

In PostgreSQL, We can also generate Database full script using Backup Option of PGAdmin tool.
When we require to create a fresh database without data, mostly we require to generate Database Schema Script.
We have one PLAIN text database backup type in PostgreSQL, which creates .sql file including Schema + Data.

Below are screenshots to generate Database Script using PGAdmin Tool:

Step 1:
Right click on the database and select backup option.
In backup dialog box, select PLAIN backup type and give a file name.

Step 2:
Default PLAIN text backup included both Database Schema + Data script.
In case of any changes, go to dump option tab and select data or schema only.

Step 3:
If you want to filter your objects, go to object tab and check or unchecked objects from the list.