Skip to content

Columnstore Index: Parallel load into clustered columnstore index from staging table

'SQL Server has supported parallel data load into a table using BCP, Bulk Insert and SSIS. The picture below shows a typical configuration of a Data Warehouse where data is loaded from external files either using BCP or SSIS. SQL Server supports parallel data load.

bcp

Another common scenario of loading data is via a staging table. Data is first loaded into one or more staging tables where it is cleansed and transformed and then it is moved to the target table as shown below. 

bcp-staging

While this works well, one challenge is that though loading data into staging tables can done in parallel, the data load from staging table into target tables is single-threaded which can slowdown the overall ETL process. To give you an extreme example, let us say you are running on a 64-core machine and the last step in your ETL is to move data from staging table to target table. If INSERT is single threaded, you will notice that only one of 64 CPUs are being used. Ideally, you want all 64 cores to be used for faster migration of data to target table.  With SQL Server 2016, you can move data from staging table into a target table in parallel which can reduce the overall data load time significantly.'...

https://blogs.msdn.microsoft.com/sqlserverstorageengine/2016/07/19/columnstore-index-parallel-load-into-clustered-columnstore-index-from-staging-table/

Trackbacks

Keine Trackbacks

Kommentare

Ansicht der Kommentare: Linear | Verschachtelt

Noch keine Kommentare

Die Kommentarfunktion wurde vom Besitzer dieses Blogs in diesem Eintrag deaktiviert.

Kommentar schreiben

Standard-Text Smilies wie :-) und ;-) werden zu Bildern konvertiert.

Um maschinelle und automatische Übertragung von Spamkommentaren zu verhindern, bitte die Zeichenfolge im dargestellten Bild in der Eingabemaske eintragen. Nur wenn die Zeichenfolge richtig eingegeben wurde, kann der Kommentar angenommen werden. Bitte beachten Sie, dass Ihr Browser Cookies unterstützen muss, um dieses Verfahren anzuwenden.
CAPTCHA

Formular-Optionen