476 CHAPTER 36 Understated changes in SQL Server 2005 replication Reading the text of hidden replication stored procedures Consider the case when you need to examine the text of a replication system stored procedure You might want to see this text for several reasons Perhaps you want to perform the same actions as the SQL Server Management Studio (SSMS) GUI, but hope to gain programmatic control of the process for yourself Perhaps you want to get a better understanding of what happens under the hood in order to increase your replication expertise and troubleshooting ability Whatever the reason, usually you will need to know the name of the system stored procedure and then use sp_helptext or the OBJECT_DEFINITION function to see the whole procedure definition For some of the replication stored procedures, though, you will find that the text is hidden and these two methods will not work For example, if you try the following code in a normal query window, you will have NULL returned: SELECT OBJECT_DEFINITION (OBJECT_id('sys.sp_MSrepl_helparticlecolumns')) On the other hand, if you use the dedicated administrator connection (DAC), you will be able to access the underlying text of the procedure The process is pretty straightforward and is shown here: Enable remote access to the DAC: sp_configure 'remote admin connections', 1; GO RECONFIGURE; GO Connect to the server using the DAC Use a query window to connect to yourservername by using ADMIN:yourservername in the server name section (or use the sqlcmd command-prompt utility with the -A switch) Execute the script: SELECT OBJECT_DEFINITION ( OBJECT_id('sys.sp_MSrepl_helparticlecolumns') ) You should find the procedure text returned as expected, and if you are on a production system, don’t forget to close the DAC connection when you are done with it! Creating snapshots without any data—only the schema When we look in BOL at the definition of a replication stored procedure or a replication agent, we find that the permitted values for the parameters are all clearly listed But it occasionally becomes apparent that there are other acceptable values that have never been documented The exact number of these hidden parameters is something we’ll never know, and in all cases they will be unsupported for the general public Even so, sometimes they start being used and recommended prior to documentation, usually in order to fix a bug A case in point is the sp_addpublication procedure, in which there is now the acceptable value of database snapshot for the @sync_method Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross Undocumented or partially documented changes in behavior 477 This value was for some time known about, undocumented and yet used, but it now exists in fully documented (and supported) form in BOL The usual caveats apply if you decide to use any such workaround; you must take full responsibility, and any such modifications are completely unsupported Another example that exists in the public domain but is not yet in BOL is also available If your distributor is SQL Server 2005, the Snapshot Agent has an undocumented /NoBcpData switch that will allow you to generate a snapshot without any BCP data This can be useful when you need to (quickly) debug schema problems generated on initialization You can access the command line for running the Snapshot Agent from SSMS as follows: Expand the SQL Server Agent node Expand the Jobs node Double-click on the Snapshot Agent job, which typically has a name of the form ___ (for example, Paul-PCTestPub-TestPublication-1) You’ll know if this is the correct job because the category will be listed as REPL-Snapshot Select Steps from the left pane Select the second Run Agent step, and click the Edit button to open it You should see the command line in the Command text box Once you have added the /NoBcpData parameter to the command line, as shown in figure 1, click OK in the Job Step dialog box and click OK again in the Job dialog box to make sure that the change is committed The /NoBcpData switch tells the Snapshot Agent to create empty BCP files instead of bulk-copying data out from the published tables Figure In the Snapshot Agent’s job step, the unofficial (unsupported!) /NoBcpData is entered Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross 478 CHAPTER 36 Understated changes in SQL Server 2005 replication Some changed replication defaults Many replication defaults changed between SQL Server 2000 and SQL Server 2005—far too many to cover in this section Most of the new defaults are obvious and self-explanatory, but occasionally some of the following changes catch people out ROW-LEVEL CONFLICT DETECTION In SQL Server 2000, column-level conflict detection was the default for merge replication, and this has changed to row-level conflict detection in SQL Server 2005 Which one is correct for your business is something only you can determine, but if you previously left this setting alone and intend to the same now, you might find an unexpected set of records in the conflict viewer NEW MERGE IDENTITY RANGE MANAGEMENT The following changes to identity range management for merge publications have been introduced in SQL Server 2005: ƒ Range allocation is automatic In SQL Server 2005 merge publications, the article identity range management is set to automatic by default In SQL Server 2000, the default identity range management was manual What is the difference? Automatic range management ensures that each subscriber is reseeded with its own identity ranges without any extra configuration, whereas manual means that you will need to change either the seed or the increment of the identity range on each subscriber to avoid conflicts with the publisher If you previously relied on leaving this article property alone and chose to manually administer the identity range, beware because a range of 1,000 values will have already been allocated to each of your subscribers ƒ Default range sizes have increased The publisher range has changed from 100 to 10,000, and the subscriber range size has increased from 100 to 1,000 ƒ Overflow range is allocated The merge trigger code on the published article implements an overflow range that is the same size as the normal range This means that by default you will have two ranges of 1,000 values allocated to a subscriber The clever part is that the overflow range is automatically allocated by the merge insert trigger and therefore doesn’t require a connection to the publisher However, the reseeding performed in the trigger is restricted to those cases where a member of the db_owner role does the insert ƒ The threshold parameter is no longer used Although it appears in the article properties dialog box much the same as in SQL Server 2000, the threshold parameter only applies to subscribers running SQL Server Mobile or previous versions of SQL Server “NOT FOR REPLICATION” TREATMENT OF IDENTITY COLUMNS Identity columns have a new default behavior These columns are automatically marked as Not for Replication (NFR) on the publisher and are transferred with the identity NFR property intact at the subscriber This retention of the NFR property applies to both transactional and merge replication Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross Undocumented or partially documented changes in behavior 479 Why might this be a useful change? First, it means that you don’t need to wade through all the tables before creating the publication in order to manually set each identity column as NFR This is a huge improvement because the method used in SQL Server 2000 by Enterprise Manager to set the NFR attribute involved making whole (time-consuming) copies of the table data It also means that if you are using transactional replication as a disaster recovery solution, there is now one less hoop you will need to jump through on failover because you don’t have to change this setting on each table at the subscriber That particular part of your process can now be removed (If you are now thinking that it is not possible in T-SQL to directly add the NFR attribute to an existing identity column, please take a look inside the sp_identitycolumnforreplication system stored procedure, because this is the procedure that marks the identity column as NFR.) DEFERRED UPDATE TRACE FLAGS For transactional replication, you might be using deferred update trace flags unnecessarily In SQL Server 2000, updates to columns that not participate in a unique key constraint are replicated as updates to the subscriber unless trace flag 8202 is enabled, after which they are treated as deferred updates (paired insert/deletes) On the other hand, updates to columns that participate in unique constraints are always treated as deferred updates (paired insert/deletes) unless trace flag 8207 is enabled In SQL Server 2005, all such changes are replicated as updates on the subscriber regardless of whether the columns being updated participate in a unique constraint or not PARTITIONING OF SNAPSHOT FILES The following change to a replication default is more complicated to explain, but it deals with a significant improvement that has been made to the initial snapshot process In SQL Server 2000, when an article is BCP’d to the filesystem (the distribution working folder) during the snapshot generation, there is always one file created that contains the table’s data In SQL Server 2005, when you look in the distribution work- Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross 480 CHAPTER 36 Understated changes in SQL Server 2005 replication ing folder after creating a snapshot, you might be surprised to find many such files for each article, each containing a separate part of the table data, as shown in figure Source table Articlename#1.bcp Articlename#2.bcp Figure Snapshot data from a table is now partitioned across several text files Clearly there has been a big change in the processing rules I’ll refer to this overall process of splitting data files as BCP partitioning, borrowing the term from a Microsoft developer who once pointed this out in a posting in the Microsoft Replication Newsgroup (microsoft.public.sqlserver.replication) This section explains why BCP partitioning exists, what the expected behavior is, and how to troubleshoot if it all goes wrong BCP partitioning has several benefits First, it helps in those cases where there has been a network outage when the snapshot is being applied to the subscriber In SQL Server 2000, this would mean that the complete snapshot would have to be reapplied, and in the case of concurrent snapshots, this would all have to be done in one transaction In contrast, if you have a SQL Server 2005 distributor and SQL Server 2005 subscribers, there is now much greater granularity in the process The article rows are partitioned into the separate text files, and each partition is applied in a separate transaction, meaning that after an outage, the snapshot distribution is able to continue with the partition where it left off and complete the remaining partitions For a table containing a lot of rows, this could lead to a huge saving in time Other useful side effects are that this can cause less expansion of the transaction log (assuming that the migration crosses a backup schedule or the subscriber uses the simple recovery model), and it can lead to paths of parallel execution of the BCP process for those machines having more than one processor (It is true that parallel execution existed in SQL Server 2000, but this was only for the processing of several articles concurrently and not for a single table.) Similarly, the same benefits apply when creating the initial snapshot using the Snapshot Agent Note that the –BcpBatchSize parameter of the Snapshot and Distribution Agents governs how often progress messages are logged and has no bearing at all on the number of partitions Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross More efficient methodologies 481 To disable BCP partitioning, you can add the unofficial -EnableArticleBcpPartitioning switch to the Snapshot Agent and a single data file will be produced, just like in SQL Server 2000 Why would you want to turn off such a useful feature? Well, anecdotally, things may get worse for folks who don’t start off with empty tables (archiving or roll-up scenarios) or if the CPU, disk I/O, or network bandwidth is the bottleneck in the attempt to extract more snapshot processing throughput when using BCP partitioning Finally, for those tables that expand the transaction log, some DBAs like to enable the bulk-logged recovery mode to try to minimize logging, but this will not always work when dealing with multiple partitions To ensure that there is a maximum chance of going down the bulk-logged path, you should use -MaxBcpThreads X (where X > 1) for the Distribution Agent and ensure that the target table doesn’t have any indexes on it before the Distribution Agent delivers the snapshot More efficient methodologies In the previous section, we looked at several undocumented techniques that can be used to enhance the replication behavior We’ll now look at some capabilities that are fully documented, but that are not always understood to be replacements for lessefficient methodologies Remove redundant pre-snapshot and post-snapshot scripts In SQL Server 2000 publications, we sometimes use pre-snapshot and post-snapshot scripts The pre-snapshot scripts are T-SQL scripts that run before the snapshot files are applied, whereas the post-snapshot scripts apply once the snapshot has completed Their use is often to overcome DRI (declarative referential integrity) issues on the subscriber Remember that the initialization process starts by dropping tables on the subscriber If all the tables on the subscriber originate from one publication, this is not an issue, but if there is more than one publication involved, we might have a scenario where the dropping of tables at the subscriber during initialization would be invalid because of relationships between articles originating from different publications There might also be other tables on the subscriber that are related to replicated articles and that are not themselves part of any publication Either way, we find the same DRI problem when initialization tries to drop the subscriber’s table In such cases, the pre-snapshot and post-snapshot scripts are needed—a pre-snapshot script would drop the foreign keys to allow the tables to be dropped, and a post-snapshot script would then add the foreign keys back in Such scripts are not difficult to write, but each needs to be manually created and maintained, causing (another!) maintenance headache for the DBA In SQL Server 2005 there is a new, automatic way of achieving this on initialization at the subscriber Initially, there is a call to the sys.sp_MSdropfkreferencingarticle Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross 482 CHAPTER 36 Understated changes in SQL Server 2005 replication system stored procedure, which saves the relevant DRI information to the following three metadata tables: ƒ dbo.MSsavedforeignkeys ƒ dbo.MSsavedforeignkeycolumns ƒ dbo.MSsavedforeignkeyextendedproperties Once the information is safely hived away, the foreign keys are dropped To re-add the foreign keys, the Distribution Agent calls the new sp_MSrestoresavedforeignkeys system stored procedure once the snapshot has been applied Note that all this happens automatically and requires no manual scripts to be created Take a look at your existing pre-snapshot and post-snapshot scripts If they deal with the maintenance of foreign keys, there’s a good chance they are doing work that is already done by default, in which case you’ll be able to drop the scripts entirely and remove the maintenance issue Replace merge -EXCHANGETYPE parameters In SQL Server 2005 merge replication, we can now mark articles as download-only, meaning that changes to the table are only allowed at the publisher and not at the subscriber Previously, in SQL Server 2000, we would use the -EXCHANGETYPE value to set the direction of merge replication changes This was implemented by manually editing the Merge Agent’s job step and adding -EXCHANGETYPE 1|2|3 as text When using SQL Server 2000, entering a value of -EXCHANGETYPE means that changes to a replicated article at the subscriber are not prohibited, are recorded in the merge metadata tables via merge triggers, and are subsequently filtered out when the Merge Agent synchronizes This means there may be a huge amount of unnecessary metadata being recorded, which slows down both the data changes made to the table and the subsequent synchronization process This -EXCHANGETYPE setting is not reflected directly in the GUI and is hidden away in the text of the Merge Agent’s job Despite being a maintenance headache and causing an unnecessary slowing down of synchronization, it was the only way of achieving this end, and judging by the newsgroups, its use was commonplace In SQL Server 2005, when adding an article, there is an option to define the subscriber_upload_options either using the article properties screen in the GUI or in code, like this: sp_addmergearticle @subscriber_upload_options = This parameter defines restrictions on updates made at a subscriber The parameter value of is described as “download only, but allow subscriber changes” and seems equivalent to the -EXCHANGETYPE = setting mentioned previously, but in the SQL Server 2005 case there are no triggers at all on the subscriber table Another distinction is that this setting is made at the more granular article level rather than set for the entire publication This means that although the -EXCHANGETYPE and sp_addmergearticle methods are logically equivalent, the implementation has become Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross Summary 483 much more sophisticated in SQL Server 2005 Triggers that unnecessarily log metadata at the subscriber are no longer fired; therefore both subscriber data changes and the subsequent synchronization are significantly faster Put simply, you should replace the use of EXCHANGETYPE with download-only articles! Incidentally, this setting is also implemented by a separate check box in SSMS, as shown in figure This check box does a similar job but sets the value of @subscriber_upload_options to 2, which again makes the changes download-only, but in this case any subscriber settings are prohibited and rolled back Figure Merge replication articles can be marked as download-only to prevent subscriber changes and reduce metadata Summary We have looked at many of the lesser-known replication techniques useful in SQL Server 2005 Some of these involve using parameters or procedures that are partially documented but that might help solve a particular issue Other methods are fully documented, but we have looked at how these methods can be used to replace replication techniques used in SQL Server 2000 and improve our replication implementation and reduce administration Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross 484 CHAPTER 36 Understated changes in SQL Server 2005 replication About the author Paul Ibison is a contractor SQL Server DBA in London He runs the website www.replicationanswers.com—the only site dedicated to SQL Server replication—and has answered over 6,000 questions on the Microsoft SQL Server Replication newsgroup When not working, he likes spending time with his wife and son, Ewa and Thomas, going fell-walking in the Lake District, and learning Ba Gua, a Chinese martial art Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross Summary Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark Licensed to Kerri Ross 485 ... CHAPTER 36 Understated changes in SQL Server 2005 replication Some changed replication defaults Many replication defaults changed between SQL Server 2000 and SQL Server 2005—far too many to cover... introduced in SQL Server 2005: ƒ Range allocation is automatic In SQL Server 2005 merge publications, the article identity range management is set to automatic by default In SQL Server 2000, the... properties dialog box much the same as in SQL Server 2000, the threshold parameter only applies to subscribers running SQL Server Mobile or previous versions of SQL Server “NOT FOR REPLICATION” TREATMENT