The MySQL Data Dictionary mysql_version.h is not in the source package, but it is in the mysql-devel package (the compilation process creates mysql_version.h later on) Unfortunately mysql_priv.h is only in the source package, not the mysql-devel package, so you need to download both the source and the mysql-devel packages On an abstract level, making a custom system view in the INFORMATION_SCHEMA database requires: ■ The field definitions of the system view (that is, the structure of the system view) In our example, we define a system view named MYSQL_HELLO with a field defined as HELLO VARCHAR(64) NOT NULL DEFAULT `’ ■ The function that populates the system view upon request (that is, the values in the system view) We will define a function to store the string plugin: hello, information_schema!!! into the HELLO field of our MYSQL_HELLO system view On a more technical level, to create the custom system view, you need to instantiate the ST_SCHEMA_TABLE struct and define two members One member, field_info, is an array of ST_FIELD_INFO structures, which define the fields in your system view The values in the ST_FIELD_INFO structure are defined in the source code in the sql/table.h header file and explained in Table 21-3 TABLE 21-3 ST_FIELD_INFO Structure Code Explanation Example Value const char* field_name; Field name ‘‘HELLO’’ uint field_length; In string-type fields, the maximum number of characters In other fields, the display length 64 in our example, a string-type field enum enum_field_types field_type; Field data type MYSQL_TYPE_VARCHAR int value; Field value uint field_flags; means NOT NULL and SIGNED It can be overridden by MY_I_S_MAYBE_NULL, MY_I_S_UNSIGNED, or both constants, separated by | (bitwise or) const char* old_name; Old field name ‘‘Hello’’ uint open_method; How the table is opened One of SKIP_OPEN_TABLE, OPEN_FRM_ONLY, or OPEN_FULL_TABLE SKIP_OPEN_TABLE Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 717 21 Part IV Extending Your Skills Our example contains the following code to define the custom system view: static ST_FIELD_INFO mysql_is_hello_field_info[]= { 10 {"HELLO", 64, MYSQL_TYPE_VARCHAR, 0, 0, "Hello", SKIP_OPEN_TABLE}, 11 {NULL, 0, MYSQL_TYPE_NULL, 0, 0, NULL, 0} 12 }; The first structure in the array is the field we are creating The last structure is an indication that the array is complete, and must be present We named the array mysql_is_hello_field_ info, which stands for ‘‘the field information of the hello system view in the mysql information schema.’’ The next line of code is: 13 int schema_table_store_record(THD *thd, TABLE *table); This line declares the schema_table_store_record function, which we will use later to store a row in a system view The second member of ST_SCHEMA_TABLE that we need to define is the function that populates the system view, called the fill_table: 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 int mysql_is_hello_fill_table( THD *thd , TABLE_LIST *tables , COND *cond ) { int status; CHARSET_INFO *scs= system_charset_info; TABLE *table= (TABLE *)tables->table; const char *str = "plugin: hello, information_schema!!!"; table->field[0]->store( str , strlen(str) , scs ); status = schema_table_store_record( thd , table ); return status; } In lines 15–17 we see THD *thd again, pointing to the current session, and TABLE_LIST *tables, which is an array of table instantiations of our ‘‘Hello’’ system view COND *cond is a condition that could be used by our plugin for filtering or optimization, though we will not use it in this example 718 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark The MySQL Data Dictionary The status variable is initialized as an int on line 20 At the end of the function (line 33), status is returned; a value of indicates an error, and a value of indicates success Then on line 21, CHARSET_INFO *scs is initialized with the character set information This is needed to properly store the field of our system view On line 22, TABLE *table is set as the handler to the instantiation of our system view We initialize the char *str on line 23 with the string we will store in our system view This means that when we finally install the plugin, we will see str in our system view: mysql> SELECT * FROM INFORMATION_SCHEMA.MYSQL_HELLO; + + | HELLO | + + | plugin: hello, information_schema!!! | + + row in set (0.00 sec) Lines 24–28 store the string str, its length, and the character set scs (defined on line 21) into the first field (field[0]) of the row Lines 29–32 store the row into the instantiation of our system view for the current session If the function that stores the row is successful, status gets a value of If there was an error, status gets a value of status is returned on line 33, thus giving the mysql_is_hello_fill_table function a return value of if it is successful and if there was an error So far we have defined a system view in an ST_FIELD_INFO array and a fill_table function to populate the system view We now need to create a plugin function to use these: 35 36 37 38 39 40 41 static int mysql_is_hello_plugin_init(void *p) { ST_SCHEMA_TABLE *schema= (ST_SCHEMA_TABLE *)p; schema->fields_info= mysql_is_hello_field_info; schema->fill_table= mysql_is_hello_fill_table; return 0; } The plugin_init function initializes the plugin as an INFORMATION SCHEMA plugin with the fields_info and fill_table we defined previously Because we have a simple plugin_init function, we have a simple plugin_deinit function as well: 42 43 44 45 static int mysql_is_hello_plugin_deinit(void *p) { return 0; } In our example there is nothing that needs to be done during the plugin_deinit, so we simply return 0, indicating the function was successful In a more complex example, there may be memory or other resources allocated in the plugin_init function that should be deallocated in the plugin_deinit function Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 719 21 Part IV Extending Your Skills At this point, we still have not created the plugin However, we have most of the parts we need to make an INFORMATION_SCHEMA plugin, and can now create the code to define the plugin itself: 46 47 48 49 50 struct st_mysql_information_schema mysql_is_hello_plugin= { MYSQL_INFORMATION_SCHEMA_INTERFACE_VERSION }; mysql_declare_plugin(mysql_is_hello) { MYSQL_INFORMATION_SCHEMA_ PLUGIN, /* type constant */ 51 &mysql_is_hello_plugin, /* type descriptor */ 52 "MYSQL_HELLO", /* Name */ 53 "Roland Bouman (http:// rpbouman.blogspot.com/)", /* Author */ _ 54 "Says hello.", /* Description */ 55 PLUGIN_LICENSE_GPL, /* License */ 56 mysql_is_hello_plugin_init, /* Init function */ 57 mysql_is_hello_plugin_deinit, /* Deinit function */ 58 0x0010, /* Version (1.0) */ 59 NULL, /* status variables */ 60 NULL, /* system variables */ 61 NULL /* config options */ 62 } 63 mysql_declare_plugin_end; Lines 46–47 define the type descriptor for the plugin, which is set as part of the plugin in line 51 Line 50 defines the type of plugin as an INFORMATION_SCHEMA plugin Lines 52–55 define metadata that appear after plugin installation in the PLUGINS system view as the PLUGIN_NAME, PLUGIN_AUTHOR, PLUGIN_DESCRIPTION, and PLUGIN_LICENSE fields, respectively Lines 56–57 point to the previously created plugin_init and plugin_deinit functions, and line 58 defines the version metadata, which corresponds to the value of PLUGIN_VERSION in the PLUGINS system view Lines 59–61 are pointers to structures containing the status variables, system variables, and configuration options Our plugin does not have any of these, so we define them as NULL The entirety of the code is: #include #include #include 720 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark The MySQL Data Dictionary #include #include #include #include static ST_FIELD_INFO mysql_is_hello_field_info[]= { {"HELLO", 64, MYSQL_TYPE_VARCHAR, 0, 0, "Hello", SKIP_OPEN_TABLE}, {NULL, 0, MYSQL_TYPE_NULL, 0, 0, NULL, 0} }; int schema_table_store_record(THD *thd, TABLE *table); int mysql_is_hello_fill_table( THD *thd , TABLE_LIST *tables , COND *cond ) { int status; CHARSET_INFO *scs= system_charset_info; TABLE *table= (TABLE *)tables->table; const char *str = "plugin: hello, information_schema!!!"; table->field[0]->store( str , strlen(str) , scs ); status = schema_table_store_record( thd , table ); return status; } static int mysql_is_hello_plugin_init(void *p) { ST_SCHEMA_TABLE *schema= (ST_SCHEMA_TABLE *)p; schema->fields_info= mysql_is_hello_field_info; schema->fill_table= mysql_is_hello_fill_table; return 0; } static int mysql_is_hello_plugin_deinit(void *p) { return 0; } Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 721 21 Part IV Extending Your Skills struct st_mysql_information_schema mysql_is_hello_plugin= { MYSQL_INFORMATION_SCHEMA_INTERFACE_VERSION }; mysql_declare_plugin(mysql_is_hello) { MYSQL_INFORMATION_SCHEMA_ PLUGIN, &mysql_is_hello_plugin, /* type constant */ /* type descriptor */ /* Name */ /* /* /* /* /* /* /* /* /* */ */ */ */ */ */ */ */ */ "MYSQL_HELLO", "Roland Bouman (http:// rpbouman.blogspot.com/)", "Says hello.", PLUGIN_LICENSE_GPL, mysql_is_hello_plugin_init, mysql_is_hello_plugin_deinit, 0x0010, NULL, NULL, NULL Author Description License Init function Deinit function Version (1.0) status variables system variables config options } mysql_declare_plugin_end; Compiling the plugin Now that the code has been created, we need to compile the plugin as a dynamic shared object and then load it into mysqld In our example we will save the source code in a file named mysql_is_hello.cc, and use g++ to compile the source Table 21-4 shows the compile flags we will use and their meanings So the complete compile statement is: shell> g++ -DMYSQL_DYNAMIC_PLUGIN –Wall –shared \ -I/path/to/mysql-6.0.x-source/include \ -I/path/to/mysql-6.0.x-source/sql \ -I/usr/include/mysql \ -o mysql_is_hello.so mysql_is_hello.cc The include paths were not specified for stdlib.h and ctype.h, because they are likely in your default include path On our machines, we found these files in /usr/include, which was in the default include path If you get errors such as: 722 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark The MySQL Data Dictionary error: stdlib.h: No such file or directory error: ctype.h: No such file or directory you should specify the include path for stdlib.h and ctype.h with another -I flag Some architectures, such as AMD64, require shared libraries to be compiled with the –fPIC flag If your architecture requires this, you will see output similar to the following when you try to compile: /usr/bin/ld: /tmp/ccNXOEqH.o: relocation R_X86_64_32S against `a local symbol’ can not be used when making a shared object; recompile with –fPIC /tmp/ccNXOEqH.o: could not read symbols: Bad value collect2: ld returned exit status In this case, simply add –fPIC to the list of compile flags and try to compile again TABLE 21-4 g++ Compile Flags for Compiling a Plugin as a Dynamic Shared Object Compile Flag Meaning -DMYSQL_DYNAMIC_ PLUGIN -D indicates a constant; the constant is MYSQL_DYNAMIC_PLUGIN, which is required to compile the plugin as a dynamic shared object for mysqld -Wall -W indicates warnings should be shown; all specifies showing all warnings -shared Compile as a shared, dynamically linked library -I/path/to/mysql6.0.x-source/include The include path for the my_global.h, my_dir.h, and mysql/plugin.h header files -I/path/to/mysql6.0.x-source/sql The include path for the mysql_priv.h header file -I/usr/include/mysql The include path for the mysql_version.h header file -o mysql_is_hello.so The output file The extension so is traditionally used for shared objects If all goes well, a dynamic shared plugin will be created with a filename of mysql_is_ hello.so Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 723 21 Part IV Extending Your Skills Installing the plugin MySQL plugins are installed in mysqld using a MySQL extension to SQL Thus, mysqld must be running, and no downtime is required to install a plugin For security purposes, mysqld only looks for plugin libraries in the directory specified by the system variable plugin_dir: mysql> SHOW VARIABLES LIKE ’plugin_dir’; + -+ -+ | Variable_name | Value | + -+ -+ | plugin_dir | /usr/lib64/mysql/plugin | + -+ -+ row in set (0.01 sec) For our example, we make sure the plugin directory exists, copy the plugin to the mysqld plugin directory, and ensure that the directory and plugin have appropriate permissions so mysqld can load the plugin: shell> mkdir /usr/lib64/mysql/plugin/ shell> cp mysql_is_hello.so /usr/lib64/mysql/plugin shell> sudo chown –R mysql:mysql /usr/lib64/mysql/plugin/ Now we use the INSTALL PLUGIN statement to install our plugin The syntax of INSTALL PLUGIN is: INSTALL PLUGIN plugin_name SONAME ’plugin_library’; The plugin_name was set in the plugin metadata (on line 52 of our code) The plugin_library is the name of the compiled plugin file (the complete path to the plugin file is plugin_dir/plugin_library): mysql> INSTALL PLUGIN MYSQL_HELLO SONAME ’mysql_is_hello.so’; Query OK, rows affected (0.00 sec) And our plugin has successfully been installed: mysql> SELECT * FROM INFORMATION_SCHEMA.MYSQL_HELLO; + + | HELLO | + + | plugin: hello, information_schema!!! | + + row in set (0.00 sec) We can now see our plugin in the mysql.plugin system table and the INFORMATION_ SCHEMA.PLUGINS system view: mysql> SELECT * FROM mysql.plugin -> WHERE name=’MYSQL_HELLO’\G 724 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark The MySQL Data Dictionary *************************** row *************************** name: MYSQL_HELLO dl: mysql_is_hello.so row in set (0.00 sec) mysql> SELECT * FROM INFORMATION_SCHEMA.PLUGINS -> WHERE PLUGIN_NAME=’MYSQL_HELLO’\G *************************** row *************************** PLUGIN_NAME: MYSQL_HELLO PLUGIN_VERSION: 0.21 PLUGIN_STATUS: ACTIVE PLUGIN_TYPE: INFORMATION SCHEMA PLUGIN_TYPE_VERSION: 60006.0 PLUGIN_LIBRARY: mysql_is_hello.so PLUGIN_LIBRARY_VERSION: 1.0 PLUGIN_AUTHOR: Roland Bouman (http://rpbouman.blogspot.com/) PLUGIN_DESCRIPTION: Says hello PLUGIN_LICENSE: GPL row in set (0.00 sec) To uninstall the plugin, run UNINSTALL PLUGIN plugin_name: mysql> UNINSTALL PLUGIN MYSQL_HELLO; Query OK, rows affected (0.00 sec) The plugin is loaded when the INSTALL PLUGIN is run If you want to change the plugin you have to uninstall the plugin, change the plugin file, and re-install the plugin If the plugin file changes or is removed while installed on a running server, mysqld will most likely crash Summary This chapter described the information contained in the MySQL data dictionary, including: ■ All of the metadata in the system views of the INFORMATION_SCHEMA database ■ Many SHOW statements (see Chapter for the remainder of the SHOW statements that show metadata) ■ Many of the system tables in the mysql database ■ Creating custom metadata as INFORMATION_SCHEMA plugins Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 725 21 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark A MySQL Proxy Server version: 6.0.8-alpha-community-log MySQL Community Server (GPL) Type ’help;’ or ’\h’ for help Type ’\c’ to clear the buffer mysql> show processlist\G *************************** row *************************** Id: User: root Host: localhost:1583 db: NULL Command: Query Time: State: NULL Info: show processlist row in set (0.00 sec) mysql> shell> mysql -u root -p -h 127.0.0.1 -P4040 Password: Welcome to the MySQL monitor Commands end with ; or \g Your MySQL connection id is Server version: 6.0.8-alpha-community-log MySQL Community Server (GPL) Type ’help;’ or ’\h’ for help Type ’\c’ to clear the buffer mysql> show processlist\G *************************** row *************************** Id: User: root Host: localhost:1583 db: NULL Command: Sleep Time: 58 State: Info: NULL *************************** row *************************** Id: User: root Host: localhost:1588 db: NULL Command: Query Time: State: NULL Info: show processlist rows in set (0.00 sec) There is no way for a user to tell whether or not they connected via mysql-proxy or directly to the mysqld instance 752 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark MySQL Proxy Multiple backends can be specified by using more than one proxy-backend-addresses option In a real load-balancing example, you would specify that mysql-proxy should listen on localhost port 3306 and forward traffic to databases on server1.company.com port 3306, server2.company.com port 3306, and server3.company.com port 3306: mysql-proxy proxy-address=localhost:3306 \ proxy-backend-addresses=server1.company.com:3306 \ proxy-backend-addresses=server2.company.com:3306 \ -backend-addresses=server3.company.com:3306 In this setup, mysql-proxy will serve the connections in a round-robin fashion When a connection exits, the server it was connected to automatically becomes the next server to receive a connection The preceding setup will produce the results summarized in Table A-1 TABLE A-1 Simple Round-Robin Behavior of mysql-proxy with Multiple Backends Action Sequence Backend Connection Connect to localhost:3306 server1.company.com:3306 Exit server1 connection server1.company.com:3306 Connect to localhost:3306 server1.company.com:3306 Connect to localhost:3306 server2.company.com:3306 Connect to localhost:3306 server1.company.com:3306 Exit server1 connection server1.company.com:3306 Connect to localhost:3306 server2.company.com:3306 Connect to localhost:3306 server2.company.com:3306 Connect to localhost:3306 server1.company.com:3306 Exit server2 connection server2.company.com:3306 Reconnect to localhost:3306 server3.company.com:3306 Connect to localhost:3306 server1.company.com:3306 Connect to localhost:3306 server2.company.com:3306 Connect to localhost:3306 server3.company.com:3306 Connect to localhost:3306 server1.company.com:3306 Connect to localhost:3306 Connect to localhost:3306 Connect to localhost:3306 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 753 A A MySQL Proxy The mysql-proxy daemon does take into consideration whether previous connections via mysql-proxy are still alive, as shown in Table A-2 The columns at the right of the table show how many connections there are to each server after the action is taken TABLE A-2 More Complex Round-robin Behavior of Backends mysql-proxy with Multiple Action Sequence Backend Connection # server1 cxns # server2 cxns # server3 cxns Connect to localhost:3306 server1.company com:3306 0 Exit server1 connection server1.company com:3306 0 Connect to localhost:3306 server2.company com:3306 0 Connect to localhost:3306 server1.company com:3306 1 Exit server1 connection server2.company com:3306 Connect to localhost:3306 server3.company com:3306 1 Exit server2 connection server2.company com:3306 0 Connect to localhost:3306 server1.company com:3306 1 Connect to localhost:3306 server3.company com:3306 1 Exit server2 and server3 server1.company com:3306 0 Connect to localhost:3306 server2.company com:3306 1 Exit server1 connection server2.company com:3306 Connect to localhost:3306 server3.company com:3306 1 1 Connect to localhost:3306 754 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark MySQL Proxy TABLE A-2 (continued ) Action Sequence Backend Connection # server1 cxns # server2 cxns # server3 cxns Connect to localhost:3306 1 Exit server2 connection Connect to localhost:3306 1 Connect to localhost:3306 2 Connect to localhost:3306 2 Scripting Using mysql-proxy without a script allows load-balancing queries in a round-robin fashion This is useful for sending read queries to many slaves, where the slaves may or may not be up However, much of the power of mysql-proxy comes from being able to program functionality with a Lua script Lua is an interpreted language like Perl, and as such, scripts not need to be compiled Scripts can be easily edited, and mysql-proxy will re-read its script for every new connection To call mysql-proxy with a script, use the proxy-lua-script option: mysql-proxy proxy-address=localhost:3306 \ proxy-backend-addresses=server1.company.com:3306 \ proxy-backend-addresses=server2.company.com:3306 \ proxy-lua-script=/usr/local/mysql-proxy/running.lua In mysql-proxy version 0.7.0, the proxy-lua-script is cached and mysql-proxy will re-read the script when the script changes Programming mysql-proxy consists of overriding one or more of six functions: ■ connect_server() ■ read_handshake() ■ read_auth() ■ read_auth_result() ■ read_query() ■ read_query_result() Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 755 A A MySQL Proxy A client attempting to log in to mysqld first tries to connect to the server By overriding the connect_server() function, you can change the way mysql-proxy passes connections to backend servers Once mysql-proxy sends passes the connection attempt to a backend server, the backend server sends handshake information back Overriding the read_handshake() function in mysql-proxy will give access to this information, which contains the version of mysqld, the thread_id that mysqld assigns this connection, the host and port of the client and server, and a password scramble After these first two steps in the handshake, the client then sends authentication information Overriding the mysql-proxy function read_auth() gives access to the username, salted password, and default database the client sends The server attempts to authenticate the client login, sending back success or failure, which can be accessed by overriding the read_auth_result() function of mysql-proxy The previous four functions deal with the initial client connection The remaining two functions allow queries and results to be changed — these functions are read_query() and read_query_result() Running MySQL proxy If you start mysql-proxy as described previously and there is no script in /usr/local/ mysql-proxy/running.lua, clients will be able to form connections to the backend servers; however, mysql-proxy will contain the following warning on each connection and for each query sent by a client: lua_loadfile(/usr/local/mysql-proxy/running.lua) failed: cannot open /usr/local/mysql-proxy/running.lua: No such file or directory If the script exists but is empty, the client will be able to connect and no errors or warnings will occur Errors and warnings from mysql-proxy are directed to standard output Thus far you have only seen how to run mysql-proxy as an active process in the foreground It is possible to run mysql-proxy as a daemon, optionally specifying a filename where the process ID can be stored: mysql-proxy proxy-address=localhost:3306 \ proxy-backend-addresses=server1.company.com:3306 \ proxy-backend-addresses=server2.company.com:3306 \ proxy-lua-script=/usr/local/mysql-proxy/running.lua \ daemon pid-file=/var/run/mysql-proxy.pid In versions 0.7.0 and later, mysql-proxy outputs nothing to standard output by default Output only occurs when the log level is set using the following option: log-level=(error|warning|info|message|debug) Only one log level can be set, but the subsequent log levels are cumulative For example, the info level contains all errors, warnings, and informational notes 756 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark MySQL Proxy Learning Lua Basics The first thing to note is that comments in Lua are of the form: a single line comment [[ a comment that spans multiple lines ]] There are no other acceptable characters to indicate a comment, and attempting to use any others (such as #, //, /* */) will generate a warning, such as the following warning when # appears at the beginning of a line: lua_loadfile(/usr/local/mysql-proxy/running.lua) failed: /usr/ local/mysql-proxy/running.lua:1: unexpected symbol near ’#’ To print out strings, the print() function is used in combination with (the concatenation function): print("Hello, " "World!") To assign variables, the assignment operator = is used The local keyword can be used to set the scope of a variable To test values, the syntax if then elseif end is used, and == is the operator to test for equality: local numval=2 if numval>1 then print(numval " is greater than 1") elseif numval1 then print(numval " is greater than 1") elseif numval mysql –h 127.0.0.1 –P3306 we got a normal query: select @@version_comment limit Mysql> SHOW DATABASES; we got a normal query: SHOW DATABASES Mysql> USE sakila; we got a normal query: SELECT DATABASE() we got a normal query: show databases we got a normal query: show tables mysql> SELECT COUNT(*) FROM staff; we got a normal query: SELECT COUNT(*) FROM staff This example is very interesting — the client sees the normal heading, such as: Welcome to the MySQL monitor Commands end with ; or \g Your MySQL connection id is Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 759 A A MySQL Proxy Server version: 6.0.8-alpha-community-log MySQL Community Server (GPL) Type ’help;’ or ’\h’ for help Type ’\c’ to clear the buffer mysql> No queries were issued, and yet mysql-proxy shows that the client sent a query That query actually does the following: mysql> select @@version_comment limit 1; + -+ | @@version_comment | + -+ | MySQL Community Edition (GPL) | + -+ row in set (0.00 sec) So, the comment after the server version in the header is actually derived from the result of a query sent from the client to mysqld In a similar manner, the output of SHOW DATABASES and the last query in the example are expected However, the mysql-proxy output from the client sending USE sakila is not expected SELECT DATABASE() is sent by the client to change the active database A client issuing USE sakila sees: mysql> USE sakila; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with –A Database changed mysql> The table and column name completions are a feature on non-Windows clients The feature enables tab completion of table and column names The way the client knows the table and column names for tab completing is simple: it queries the server every time the active database is changed The proxy tokenizer It would be very tedious to decode all the packets sent back and forth by using string manipulation For an advanced use case such as auditing, it would be necessary to match each word to particular statements and keywords, including SELECT, INSERT IGNORE INTO, SHOW, SET and individual table and column names Luckily, the developers of mysql-proxy have created packages that make this job much simpler These packages live in the share/mysql-proxy/proxy directory (on Windows, share/proxy) These packages are Lua scripts containing modularized functions that can be used by other scripts For example, the commands.lua script defines a parse() function, which takes in an argument, and returns a Lua table structure with the following fields: ■ type ■ type_name 760 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark MySQL Proxy ■ query (when type is proxy.COM_QUERY and proxy.COM_STMT_PREPARE) ■ schema (when type is proxy.COM_INIT_DB) Instead of the previous example, the input to read_query() was manipulated by using substrings, take advantage of the parse() function: local commands = require("commands") function read_query(packet) local cmd = commands.parse(packet) if cmd.type == proxy.COM_QUERY then print("we got a normal query: " cmd.query) end end Line requires the commands.lua file and assigns the contents to an object called commands Line defines the function we are overriding Line assigns the object cmd as the output of commands.parse(packet), which is a Lua table structure Line uses the type field to check what type of statement the client sent Previously, the awkward string.byte(packet) was used to retrieve the type of statement; now you can use the cleaner and more intuitive cmd.type Similarly, line uses cmd.query instead of string.sub(packet, 2) in the print statement Lines and end the if statement and the function definition, respectively Lua Packages M odular code is extremely useful Lua has a package.path structure that stores the paths to check when using packaged code If mysql-proxy outputs errors starting with: /usr/local/mysql-proxy/running.lua:1: module ’proxy.commands’ not found: no field package.preload[’proxy.commands’] no file ’./proxy/commands.lua’ you may need to change the package.path value For example, if your installation is in /usr/local/mysql-proxy, and commands.lua is in /usr/local/mysql-proxy/ share/mysql-proxy/proxy/commands.lua, add this to the top of your Lua script: package.path = package.path ";/usr/local/mysql-proxy/share/mysql-proxy/? lua" local commands = require("proxy.commands") The first line assigns package.path to be the old value of package.path concatenated with another path As you may have guessed, the ? in the package.path string is a substitution operator This is why the second line requires commands instead of commands.lua The other reason for this is that the character is used as a directory indicator If you had specified: package.path = package.path ";/usr/local/mysql-proxy/share/mysql-proxy/ proxy" local commands = require("commands.lua") you would have seen errors that there is no such file as commands/lua Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 761 A A MySQL Proxy Getting the command type is one useful task; being able to figure out which tables are being used, what kind of query (SELECT, INSERT, UPDATE, DELETE, SHOW, etc.) is another large task The developers of mysql-proxy built in a tokenizer that divides a packet into its keywords and parts To see the tokenizer in action, run mysql-proxy with the Lua script tutorialtokenize.lua The read_query() function override calls the tokenize(packet) function The function returns a hash of tokens, which read_query() goes through, printing out the token name and text In order to use the tokenize() function, you must load the tokenizer package first with require first (see the sidebar on Lua packages for more information) For example: local tk = require("proxy.tokenizer") local tokens = tk.tokenize(query) For example, the query: SELECT COUNT(*) FROM sakila.staff WHERE email LIKE "%sakila%com"; becomes tokenized as seen in Table A-4 TABLE A-4 Sample Tokenized Query Token Name Token Value TK_SQL_SELECT SELECT TK_FUNCTION COUNT TK_OBRACE ( TK_STAR * TK_CBRACE ) TK_SQL_FROM FROM TK_LITERAL sakila TK_DOT TK_LITERAL staff TK_SQL_WHERE WHERE TK_LITERAL Email TK_SQL_LIKE LIKE TK_STRING "%sakila%com" 762 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark MySQL Proxy For the full list of tokens and their meanings, see the ‘‘MySQL Proxy Tokens’’ section later in this appendix Changing the Query Backend The backend mysqld server that mysql-proxy sends information to can be changed in the connect_server() function override and the read_query() function override The connect_server() function happens once per connection, and the read_query() function happens every time a query is sent The following is simple code to change the backend connection to the first mysqld backend in the list of proxy backends that is not in a down state: function connect_server() for i = 1, #proxy.backends local s = proxy.backends[i] if s.state ∼= proxy.BACKEND_STATE_DOWN then proxy.connection.backend_ndx = i return end end 10 end 11 12 function read_query(packet) 13 for i = 1, #proxy.backends 14 local s = proxy.backends[i] 15 16 if s.state ∼= proxy.BACKEND_STATE_DOWN then 17 proxy.connection.backend_ndx = i 18 return 19 end 20 end 21 end Note that both functions contain the same code Both of these function overrides may or may not be necessary, depending on your application In cases where neither persistent connections nor connection pools are being used, the ratio of queries:connection is 1:1 When the number of queries is greater than the number of connections, you are in a scenario where there is a query sent to an existing connection It is in this case that you want to override both the connect_server() function and the read_query() function In lines and 13, #proxy.backends is the length of the proxy.backends array This is the number of backends that were specified by the backend-proxy-addresses in the mysql-proxy configuration Lua starts numbering its indexes at 1, not 0, so the for loop starts at and continues until i has the value of the length of the array Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 763 A A MySQL Proxy For each backend, lines and 16 evaluate whether or not that backend is in a down state (unreachable) The not equals operator in Lua is ∼= If the backend is not down, this connection’s backend is set to that backend, and the function returns Otherwise, the for loop continues Setting the proxy backend for a connection is as simple as the assignment on lines and 17: proxy.connection.backend_ndx = i Using more complicated Lua scripts, the read_query() function can be overridden, starting with a query tokenizer, using some part of the query to decide which backend the query should go to, and ending with setting the backend of the connection This is how read/write splitting and arbitrary partitioning can be achieved with mysql-proxy Changing and Injecting Queries By overriding the functions read_query() and read_query_result(), the queries sent to the mysqld backend server and the output that is returned can be examined and modified, if desired A sample Lua script to perform query injection with mysql-proxy overrides the read_query() function: local commands = require("commands") function read_query(packet) local cmd = commands.parse(packet) if cmd.type == proxy.COM_QUERY then print("we got a normal query: " cmd.query) proxy.queries:append(1, packet) proxy.queries:append(2, string.char(proxy.COM_QUERY) "SELECT NOW()" ) return proxy.PROXY_SEND_QUERY end 10 end In mysql-proxy versions 0.7.0 and greater, the append() and prepend() syntax has been changed Instead of: proxy.queries:prepend(num,packet) proxy.queries:append(num,packet) a third argument has been added to improve performance The syntax now is: proxy.queries:prepend(num, packet, {resultset_is_needed = true }) proxy.queries:append(num, packet, {resultset_is_needed = true }) This third argument is needed to send the result set back Lines 1-5 are familiar; they were explained in ‘‘The Proxy Tokenizer’’ section earlier in this appendix Lines through 8, however, are new Lines and add values to the 764 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark MySQL Proxy proxy.queries array The proxy.queries array is the array of queries that mysql-proxy sends to the backend Line pushes the id value and the existing query packet as the first item in the proxy.queries array Line pushes the id value and a new query as the second item in the proxy.queries array Line sends the modified proxy.queries array to the backend mysqld server Note that the id values can be any number you choose The order of proxy.queries is determined by the order in which items are appended to the array (or prepended with the proxy.queries:prepend function) If lines and had been: proxy.queries:append(2, packet) proxy.queries:append(1, string.char(proxy.COM_QUERY) "SELECT NOW()" ) the order of the queries in proxy.queries would still have been the same — the original query would still be first, and the new query, SELECT NOW(), would be second The first query has an id value of 2, and the second query has an id value of 1, but the id values are not what determine the order of the queries This can be confusing! So at this point, two queries were sent to mysqld instead of one, for each COM_QUERY type of statement sent through mysql-proxy The results still have to be handled on the way back — the client only sent one query, so it only expects one result The result must be intercepted to make sure that the client only receives one result set back, instead of two result sets To intercept the query results, override the read_query_result() function: 11 12 13 14 15 16 17 18 19 20 function read_query_result(inj) print("injected result-set: id = " inj.id) if (inj.id == 2) then for row in inj.resultset.rows print("injected query returned: " row[0]) end return proxy.PROXY_IGNORE_RESULT end end The read_query_result() function is called for each result set that is sent back by the server In this example, the read_query() function was called once, the read_query_ result() function will be called twice — first, for the result of the original query, and second, for the result of the injected query On line 12, you print the id value that you assigned when appending the queries Again, this is just the id value and has no bearing on the order of the results The order of the results is determined by the order of the queries in the proxy.queries array, not by the id values of the queries On line 14, the script starts to deal with the case that your query is the injected query, whose id is Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark 765 A A MySQL Proxy On lines 15 through 17, the first field of each row in the result set is printed On line 18, mysql-proxy is told not to send along this result set; this is still part of the case where the query has an id of Line 19 closes the case that this query has an id of Line 20 ends the read_query_result() function — the result set is sent by default if there is no proxy.PROXY_IGNORE_RESULT value returned Overriding read_query() allows us to change the queries sent Lua can change the query that is sent — change the original query itself or add more queries to be sent Overriding read_query_result() allows the results sent to be changed, which can be used in conjunction with injected queries to handle multiple results However, read_query_result() can also be overridden without having changed the query sent to the mysqld backend Thus, the result(s) sent back to the client can be changed even if the original query sent was not changed Understanding MySQL Proxy Internals There are several internal structures that mysql-proxy has control over Using a Lua script to read, and in some cases write to, these internal structures is what allows you to change mysql-proxy default behavior For example, to change mysql-proxy from its default round-robin load-balancing behavior, you would use a Lua script to override the connect_server() function, and use the internal structure proxy.connection.backend_ndx to change the backend server that mysql-proxy will forward the connection to The top-level internal structure is proxy, which is a container for all the internal structures used by mysql-proxy The second-level internal structures are backends, connection, servers, queries, and global The second level also contains many constants used for comparison, as shown in Table A-5 TABLE A-5 MySQL Proxy Constants Constant Name Context Meaning BACKEND_STATE_DOWN proxy.backends [x].state The backend server is down BACKEND_STATE_UNKNOWN proxy.backends [x].state The backend server is in an unknown state BACKEND_STATE_UP proxy.backends [x].state The backend server is up BACKEND_TYPE_RO proxy.backends [x].type The backend server was defined with proxy-readonly-address 766 Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark ... itself: 46 47 48 49 50 struct st _mysql_ information_schema mysql_ is_hello_plugin= { MYSQL_ INFORMATION_SCHEMA_INTERFACE_VERSION }; mysql_ declare_plugin (mysql_ is_hello) { MYSQL_ INFORMATION_SCHEMA_ PLUGIN,... st _mysql_ information_schema mysql_ is_hello_plugin= { MYSQL_ INFORMATION_SCHEMA_INTERFACE_VERSION }; mysql_ declare_plugin (mysql_ is_hello) { MYSQL_ INFORMATION_SCHEMA_ PLUGIN, &mysql_ is_hello_plugin,... static ST_FIELD_INFO mysql_ is_hello_field_info[]= { {"HELLO", 64, MYSQL_ TYPE_VARCHAR, 0, 0, "Hello", SKIP_OPEN_TABLE}, {NULL, 0, MYSQL_ TYPE_NULL,