Chapter 8: Data Buffering and Transactions 273 if the order could not be added. A single transaction is not enough, but a pair of nested transactions will meet the bill very well as illustrated below: *** Start the outermost ('Wrapper') transaction BEGIN TRANSACTION *** Update the Customer table first llTx1 = TableUpdate( 1, .F., 'customer' ) IF llTx1 *** Customer table updated successfully *** Start the second, 'inner' transaction for Orders BEGIN TRANSACTION llTx2 = TableUpdate( 1, .F., 'orderheader' ) IF llTx2 *** Orders Updated, now try details llTx2 = TableUpdate( 2, .F., 'orderdetails' ) IF llTx2 *** Both Order tables updated successfully *** So commit the orders transaction END TRANSACTION ELSE *** Order Detail update failed *** Roll back entire orders transaction ROLLBACK ENDIF ELSE *** Order Header update failed - no point in trying details ROLLBACK ENDIF *** But the customer update had already succeeded, so commit END TRANSACTION ELSE *** Customer update failed - no point in proceeding ROLLBACK ENDIF This code may look a little strange at first sight but does emphasize the point that BEGIN…END TRANSACTION does not constitute a control structure. Notice that there are two starting commands, one for each transaction and two 'Commit' commands, one for the customer table and one for the pair of order tables. However there are three rollback commands. One for the outer transaction but two for the inner transaction to cater to the fact that either table involved might fail. The logic gets a little more tricky as more tables are involved but the principles remain the same. However, when many tables are involved, or if you are writing generic routines to handle an indeterminate number of tables at each level, it will probably be necessary to break the transactions up into separate methods to handle Update, Commit and Rollback functions 274 1001 Things You Always Wanted to Know About Visual FoxPro and to use the TXNLEVEL() function to keep track of the number of transactions. (Remember that Visual FoxPro is limited to five simultaneous transactions.) Some things to watch for when using buffering in applications Cannot use OLDVAL() to revert a field under table buffering You may have wondered, when reading the discussion of conflict resolution earlier in this chapter, why we did not make use of the OLDVAL() function to cancel a user's changes in the same way that we used the value returned by CURVAL() to clear conflicts in different fields. The answer is simply that we cannot do it this way when using any form of table buffering if the field is used in either a Primary or Candidate index without getting a "Uniqueness of index <name> is violated" error. This is an acknowledged bug in all versions of Visual FoxPro and, since the work-around offered by Microsoft is to use the TableRevert() function instead, it seems unlikely to us that it will ever be fixed. However, this does not seem to be an entirely satisfactory solution because TableRevert() cannot operate at anything other than "Row" level and so undoing a change to a key field without losing any other changes in the same record requires a little more thought. The best solution that we have found is to make use of the SCATTER NAME command to create an object whose properties are named the same as the fields in the table. The values assigned to the object's properties are the current values from the record buffer and we can change the property values using a simple assignment. The following program illustrates both the problem and the solution: ********************************************************************** * Program : ShoOVal.prg * Compiler : Visual FoxPro 06.00.8492.00 for Windows * Abstract : Illustrates problem with using OldVal() to revert field * : which is used in a Candidate key * : Ref: MS Knowledgebase PSS ID Number: Q157405 ********************************************************************** *** Create and populate a sample table CREATE TABLE sample ( Field1 C(5) UNIQUE, Field2 N(2) ) INSERT INTO sample ( Field1, Field2 ) VALUES ( "one ", 1 ) INSERT INTO sample ( Field1, Field2 ) VALUES ( "two ", 2 ) INSERT INTO sample ( Field1, Field2 ) VALUES ( "three", 3 ) *** Force into Table Buffered mode SET MULTILOCK ON CURSORSETPROP( "Buffering", 5 ) *** FIRST THE PROBLEM *** Change key field GO TOP REPLACE field1 WITH "four", field2 WITH 4 SKIP SKIP -1 *** Revert value using OldVal() REPLACE field1 WITH OLDVAL( "Field1" ) Chapter 8: Data Buffering and Transactions 275 SKIP SKIP -1 *** You now get a "Uniqueness of index FIELD1 is violated" error message *** Click IGNORE and revert the table - loses changes in all fields! TableRevert(.T.) *** NOW THE SOLUTION *** Repeat the Replace GO TOP REPLACE field1 WITH "four", field2 WITH 4 SKIP SKIP -1 *** Scatter the fields to an Object SCATTER NAME loReverter *** Revert the Key Field value loReverter.Field1 = OLDVAL( 'Field1' ) *** Revert the row in the table TableRevert(.F.) *** Gather values back GATHER NAME loReverter SKIP SKIP-1 *** NOTE: No error, and the change in Field2 is retained *** Confirm the reversion TableUpdate(1) BROW NOWAIT At the time of writing, the behavior outlined above was rather erratic when a character string was used as the candidate key. For example, if you REPLACE field1 WITH "seven" you do not get an error at all! However, there is nothing magical about the string "seven" as several other values were found that did not cause an error, but we could not discern a pattern in, or formulate any logical explanation for, the observed behavior. Gotcha! Row buffering and commands that move the record pointer We noted earlier that changes to a record in a row buffered table are automatically committed by Visual FoxPro whenever the record pointer for that table is moved. This is implicit in the design of Row Buffering and is entirely desirable. What is not so desirable is that it is not always obvious that a specific command is actually going to move the record pointer and this can lead to unexpected results. For example, it is predictable that issuing a SEEK or a SKIP command is going to move the record pointer and therefore we would not normally allow such a command to be executed without first checking the buffer mode and, if row buffering is in force, checking for uncommitted changes. 276 1001 Things You Always Wanted to Know About Visual FoxPro Similarly you would expect that using a GOTO command would also move the record pointer if the specified record was not already selected. However, GOTO always moves the record pointer, even if you use the GOTO RECNO() form of the command (which keeps the record pointer on the same record) and so will always attempt to commit pending changes under row buffering. In fact, any command which can take either an explicit record number or has a Scope ( FOR or WHILE ) clause is going to cause the record pointer to move and is, therefore, a likely cause of problems when used in conjunction with Row Buffering. There are many such commands in Visual FoxPro including the obvious like SUM , AVERAGE and CALCULATE as well as some less obvious ones like COPY TO ARRAY and UNLOCK. This last is particularly sneaky. What it means is that if you are using explicit locking with a row buffered table, then unlocking a record is directly equivalent to issuing a TableUpdate() and you immediately lose the ability to undo changes. We are not sure whether this was really the intended behavior (though we can guess!) but it does underline the points made earlier in the chapter. First, there is no real place for row buffering in an application, and second, the best way to handle locking when using buffering is to allow Visual FoxPro to do it. Chapter 9: Views in Particular, SQL in General 277 Chapter 9 Views in Particular, SQL in General "There are many paths to the top of the mountain, but the view is always the same." (Chinese Proverb) The preceding two chapters have concentrated mainly on the management and use of tables in Visual FoxPro but there is much, much more that can be done by branching out into the world of SQL in general, and Views in particular. Since we are concentrating on Visual FoxPro, this chapter deals mainly with Local Views and does not address the subject of Offline or Remote views in any detail. We hope you will still find enough nuggets here to keep you interested. Visual FoxPro views The sample code for this chapter includes a separate database (CH09.DBC) which contains the local tables and views used in the examples for this section. We have not included any specific examples that use remote views (if only because we cannot guarantee that you will have a suitable data source set up on your machine) but where appropriate we have indicated any significant differences between local and remote views. What exactly is a view? The Visual FoxPro Help file defines a view in the following terms: "A customized virtual table definition that can be local, remote, or parameterized. Views reference one or more tables, or other views. They can be updated, and they can reference remote tables." The key word here is 'definition' because a view is actually a SQL query that is stored within a database container but, unlike a simple Query (.QPR) file, a view is represented visually in the database container as if it actually were a table. It can, for all practical purposes, be treated as if it really were a table (i.e. to open a view, you simply USE it, and it can also be added to the dataenvironment of a form at design time). There are at least three major benefits to be gained from using views. First, because a view does not actually store any data persistently, it requires no permanent disk storage space. However, because the definition is stored, the view can be re-created any time it is required without the need to re-define the SQL. Second, unlike a cursor created directly by a SQL Query, a view is always updateable and can, if required, also be defined to update the table, or tables, on which it is based. Third, a view can be based upon local (VFP) 278 1001 Things You Always Wanted to Know About Visual FoxPro tables, or can use tables from a remote data source (using ODBC and a 'Connection') and can even use other views as the source for its data - or any combination of the above. How do I create a view? Visual FoxPro allows a view to be created in two ways, either visually (using the View Designer) or programmatically with the CREATE SQL VIEW command. For full details on using the View Designer, see Chapter 8: Creating Views in the Programmer's Guide. (However, remember that, like all the designers, the View Designer does have some limitations and certain types of views really do need to be created in code.) Whichever method you use, the process entails four steps: • Define the fields which the view will contain and the table(s) from which those fields are to be selected • Specify any join conditions, filters or parameters required • Define the update mechanism and criteria (if the view is to be used to update its source tables) • Name and save the view to a database container One big benefit of using the view designer to create views is that it hides the complexity of the code required and, while the code is not actually difficult, there can be an awful lot of it (even for a simple view) as the following example shows. Here is the SQL for a simple, but updateable, one-table view that lists Company Names by city for a given country as shown in the View Designer: SELECT DISTINCT Clients.clisid, Clients.clicmpy, Clients.clicity; FROM ch09!clients; WHERE Clients.clictry = ?cCountry; ORDER BY Clients.clicity, Clients.clicmpy While here is the code required to create the same view programmatically: CREATE SQL VIEW "CPYBYCITY" ; AS SELECT DISTINCT Clients.clisid, Clients.clicmpy, Clients.clicity ; FROM ch09!clients ; WHERE Clients.clictry = ?cCountry ; ORDER BY Clients.clicity, Clients.clicmpy DBSetProp('CPYBYCITY', 'View', 'UpdateType', 1) DBSetProp('CPYBYCITY', 'View', 'WhereType', 3) DBSetProp('CPYBYCITY', 'View', 'FetchMemo', .T.) DBSetProp('CPYBYCITY', 'View', 'SendUpdates', .T.) DBSetProp('CPYBYCITY', 'View', 'UseMemoSize', 255) DBSetProp('CPYBYCITY', 'View', 'FetchSize', 100) DBSetProp('CPYBYCITY', 'View', 'MaxRecords', -1) DBSetProp('CPYBYCITY', 'View', 'Tables', 'ch09!clients') DBSetProp('CPYBYCITY', 'View', 'Prepared', .F.) DBSetProp('CPYBYCITY', 'View', 'CompareMemo', .T.) DBSetProp('CPYBYCITY', 'View', 'FetchAsNeeded', .F.) Chapter 9: Views in Particular, SQL in General 279 DBSetProp('CPYBYCITY', 'View', 'FetchSize', 100) DBSetProp('CPYBYCITY', 'View', 'ParameterList', "cCountry,'C'") DBSetProp('CPYBYCITY', 'View', 'Comment', "") DBSetProp('CPYBYCITY', 'View', 'BatchUpdateCount', 1) DBSetProp('CPYBYCITY', 'View', 'ShareConnection', .F.) DBSetProp('CPYBYCITY.clisid', 'Field', 'KeyField', .T.) DBSetProp('CPYBYCITY.clisid', 'Field', 'Updatable', .F.) DBSetProp('CPYBYCITY.clisid', 'Field', 'UpdateName', 'ch09!clients.clisid') DBSetProp('CPYBYCITY.clisid', 'Field', 'DataType', "I") DBSetProp('CPYBYCITY.clicmpy', 'Field', 'KeyField', .F.) DBSetProp('CPYBYCITY.clicmpy', 'Field', 'Updatable', .T.) DBSetProp('CPYBYCITY.clicmpy', 'Field', 'UpdateName', 'ch09!clients.clicmpy') DBSetProp('CPYBYCITY.clicmpy', 'Field', 'DataType', "C(40)") DBSetProp('CPYBYCITY.clicity', 'Field', 'KeyField', .F.) DBSetProp('CPYBYCITY.clicity', 'Field', 'Updatable', .T.) DBSetProp('CPYBYCITY.clicity', 'Field', 'UpdateName', 'ch09!clients.clicity') DBSetProp('CPYBYCITY.clicity', 'Field', 'DataType', "C(15)") First, it must be said that it is not really as bad as it looks! Many of the View level settings defined here are the default values and only need to be specified when you need something set up differently. Having said that, it still remains a non-trivial exercise (just getting all of the statements correctly typed is difficult enough!). So how can we simplify things a bit? Well, Visual FoxPro includes a very useful little tool named GENDBC.PRG that creates a program to re-generate a database container (You will find it in the \VFP60\TOOLS\GENDBC\ sub- directory). Views are, as stated above, stored in a database container. So why not let Visual FoxPro do all the hard work? Simply create a temporary (empty) database container and define your view in it. Run GENDBC and you have a program file that not only documents your view, but can be run to re-create the view in your actual database container. More importantly there are, as we said, some limitations to what the designer can handle. One such limitation involves creating views that join multiple tables related to a common parent, but not to each other. The join clauses produced by the designer cannot really handle this situation and the only way to ensure the correct results is to create such views in code. Using the designer to do most of the work, and simply editing the join conditions in a PRG file, is the easiest way to create complex views. One word of caution! If you do create views programmatically, be sure that you do not then inadvertently try to modify them in the view designer because you may end up with a view that no longer does what it is meant to. We strongly recommend naming such views differently to distinguish them from views that can be safely modified in the designer. To modify a programmatically created view, simply edit the program that creates it and re-run it. When should I use a view instead of a table? Actually you never need to use a table again! You can always use a view, even if that view is simply an exact copy of a single table. We will cover this point later in the chapter (see the section on Scalability). However, there are certainly some occasions when we think that a view should be used rather than using tables directly. The first, and probably the most obvious, is when creating reports that require data from related tables. While the Visual FoxPro Report Writer is quite a flexible tool, it is not (in our opinion) easy to use when trying to work with multiple tables. A single view can reduce a 280 1001 Things You Always Wanted to Know About Visual FoxPro complex relational structure to a "flat file" which is easily handled by the report writer, making the task of setting up reports that use grouped data much easier. Another, perhaps less obvious use is that some controls, like grids and list or combo boxes, often need to use look-up tables to display descriptions associated with code fields. Creating an updateable view to combine the descriptions along with the 'real' data provides a simple and efficient way of handling such tasks. Chapter 6 (Grids: The Misunderstood Controls) includes an example which uses a view for precisely this purpose - a view, used as the RecordSource for the grid, includes a description field from a lookup table, and all fields except the description are updateable. Views also provide a mechanism for accessing data in older versions of FoxPro without needing to convert the source data. If you try to add a FoxPro 2.x table to a Visual FoxPro database container, the tables become unusable by the 2.x application. In cases where you need to access the same table in both FoxPro 2.x and Visual FoxPro, a view provides the solution. Although the view must be stored within a DBC, the tables that it uses do not have to be. This ability is, of course, not limited to FoxPro tables. A view can be defined to retrieve data from any data source into Visual FoxPro, providing that an ODBC connection to that data source can be established. Once the data has been pulled into a view it can be manipulated in exactly the same way as if it were native Visual FoxPro data. By making such a "Remote" view updateable, any changes made in Visual FoxPro can be submitted to the original data source. The final occasion to use a view is when you need to obtain a sub-set of data. In this situation, creating a parameterized view is often better than simply setting a filter. In fact, when dealing with grids it is always better because grids cannot make use of Rushmore to optimize filters. For more details on using views in grids, see Chapter 6. Hang on! What is a parameterized view? Ah! Did we not mention this? A view does not always have to include all data from a table, nor do you always have to specify the exact filter condition at design time. Instead you can define a view which includes a filter condition which will be based on a parameter supplied at run time - hence the term 'Parameterized View'. A parameterized view is defined by including a filter condition that refers to a variable name, property or field in an open table which has been prefixed with a "?" as follows: WHERE Clients.clicity = ?city_to_view ; When the view is opened or re-queried, Visual FoxPro will look for the named variable and, if it finds it will simply apply the condition as specified to the SQL statement which populates the view. A simple parameterized view is included in the samples for this chapter (see lv_CpyByCity in CH09.dbc). If the named parameter is not found when the view is opened, or re-queried, a dialog box which requests a value for the parameter is displayed - like this: Chapter 9: Views in Particular, SQL in General 281 Figure 9.1 View Parameter Dialog Defining view parameters Notice that the prompt begins 'Enter a character value…'. How does Visual FoxPro know that City_To_View is a character string? The answer is that it doesn't! This particular view was created in the view designer and, when using the designer, there is an option on the 'Query' pad of the system menu ('View parameters') which allows you to define the names and data types of any parameters you specify in the definition for the view. Figure 9.2 View Parameter Definition If you are going to make use of the view parameters dialog in an application, and you define those views in the designer, always define your parameters explicitly in this dialog. (We would suggest that you use very descriptive names too!) If you do not define the parameter, then the dialog displayed to the user will not include the type of value expected and will simply state "Enter a value for City_To_View". (To do the same thing when defining a view in code merely requires (yet another) call to the DBSETPROP() function). However, in our opinion, simply using the default dialog is not a good thing to do in an application for three reasons. First, Visual FoxPro does not validate the entry the user types into the dialog (even when you pre-define the parameter and its data type), and you cannot validate that entry before it is applied either. So if the parameter is invalid, you will simply get an error. Second, the dialog itself is not really very user-friendly. After all, what is the average 282 1001 Things You Always Wanted to Know About Visual FoxPro user going to make of something that is asking for a 'View Parameter'? Finally, if you are using multiple parameters then, because Visual FoxPro can accept only one parameter at a time, the user will be presented with a series of dialogs one after another (very ugly). Using parameterized views in a form The solution is really very simple - just ensure that the appropriate parameter has been defined, is in scope and has been populated before opening or re-querying the view. There are many ways of doing this, but our preferred method is to create properties for view parameters at the Form level and then to transfer the values from those properties into the appropriate variables in whatever method is querying the view. This has two benefits. First, it ensures that the current values used by the view are available to the entire form. Second, it allows us to validate the parameters before they are passed to the view. To enable a user to specify the necessary parameters, we must provide the user with an appropriate means for entering the parameters. This might be a pop-up form or some sort of selection control on the form itself. Most importantly, we can also get multiple parameters in one operation if necessary. A form that gives an example of such a mechanism is included in the sample code for this chapter (VuParams.scx). Figure 9.3 Using a Parameterized View in a form This form uses a view (lv_adcampanes) that joins three tables, which are related as shown in Figure 9.4 and accepts two parameters, one for the "Client Name" and the other for a "Start Date." [...]... clause is, therefore, invalid: 304 1001 Things You Always Wanted to Know About Visual FoxPro SELECT SUM( invamt ) AS totalinv, ; SUM( invpaid ) AS totalpaid, ; (totalinv - totalpaid) AS invbal ; Conditionally including fields in a result set The Visual FoxPro Immediate IF function (IIF()) can be used in a query to conditionally extract data from either of two fields into a single field However, there... views is that the same view definition can be used to access either local (i.e Visual FoxPro) tables or remote (i.e back end) data The only practical difference between a local and a remote view is that the latter requires a 286 1001 Things You Always Wanted to Know About Visual FoxPro "connection" to be defined in order to enable the data source to be queried In all other respects the behavior of... ("customerid") definitions look like this: LOCAL * Props for the CUSTOMERS.customerid field DBSetProp('CUSTOMERS.customerid', 'Field', 'KeyField', T.) DBSetProp('CUSTOMERS.customerid', 'Field', 'Updatable', F.) DBSetProp('CUSTOMERS.customerid', 'Field', 'UpdateName', 'vfpdata!customers.customerid') DBSetProp('CUSTOMERS.customerid', 'Field', 'DataType', "C(5)") REMOTE * Props for the CUSTOMERS.customerid... even worse because although you can code such a parameterized view, you cannot specify the parameters through the dialog in a way that is meaningful to the SQL engine The only solution that we have been able to find (to date) is to hand code the view and use macro substitution for the parameter The SQL looks like this: 294 1001 Things You Always Wanted to Know About Visual FoxPro CREATE SQL VIEW lvCityList... that the return value does NOT tell you if any records were 284 1001 Things You Always Wanted to Know About Visual FoxPro retrieved, merely if the SQL was executed properly To get the number of matching records you still need to use _TALLY as the following code shows: USE lv_cpybycity NODATA City _to_ view = 'London' ? REQUERY() && Returns 1 ? _TALLY && Returns 6 City _to_ view = 'Peterborough' ? REQUERY()... 306 1001 Things You Always Wanted to Know About Visual FoxPro Of course, there are many other ways of converting data into a text file One that we really like (although it is only useful for files that are required in SDF format) is to make use of the new STRTOFILE() function in conjunction with the VFP application object's DataToClip method as follows: *** Open a table USE *** Copy contents to. .. current output window), it can provide you with a lot of very useful information about the way Visual FoxPro sees your queries The quickest way to get the output of SYS(3054) into a usable form is to use SET ALTERNATE to echo all screen output to a file of your choosing, as follows: SET ALTERNATE TO showplan.txt SET ALTERNATE ON If you are running a program, you can also suppress the on-screen output... 'disconnected', cursor for it The consequence is that if you change the content of the original view, the content of the alternate alias also changes To be honest, we are not sure if this is a good thing or a bad thing, but either way it does seem to be a limitation to the usefulness of the USE AGAIN option 290 1001 Things You Always Wanted to Know About Visual FoxPro It is worth noting that, for REMOTE... actual results are shown in Figure 9.9, and the number of rows returned by each type of query was: Full Join (top left): 14 Rows Inner Join (bottom left): 9 Rows Right Outer Join (top right): 10 Rows Left Outer Join (bottom right): 13 Rows 296 1001 Things You Always Wanted to Know About Visual FoxPro Figure 9.9 Different joins produce different results from the same data Notice that with every condition,... SELECT statement and use the 'AS' keyword to define a name for the column 300 1001 Things You Always Wanted to Know About Visual FoxPro Note: In fact Visual FoxPro does not require the inclusion of the 'AS' keyword at all, nor do most back-end servers In fact, some older versions of servers do not support the 'AS' keyword at all and including it in a query sent to such a back end causes an 'Invalid Column . functions 274 1001 Things You Always Wanted to Know About Visual FoxPro and to use the TXNLEVEL() function to keep track of the number of transactions. (Remember that Visual FoxPro is limited to five. NOT tell you if any records were 284 1001 Things You Always Wanted to Know About Visual FoxPro retrieved, merely if the SQL was executed properly. To get the number of matching records you still. required, also be defined to update the table, or tables, on which it is based. Third, a view can be based upon local (VFP) 278 1001 Things You Always Wanted to Know About Visual FoxPro tables, or can