Skip directly to content

Feed aggregator

DrizzleDB: Start by fixing small things http://t.co/cY8cLmH6

@DrizzleDB on Twitter - Tue, 11/08/2011 - 01:34
DrizzleDB: Start by fixing small things http://t.co/cY8cLmH6

Patrick Crews: dbqp and Xtrabackup testing

Libdrizzle News - Mon, 11/07/2011 - 22:55

So I’m back from the Percona dev team’s recent meeting.  While there, we spent a fair bit of time discussing Xtrabackup development.  One of our challenges is that as we add richer features to the tool, we need equivalent testing capabilities.  However, it seems a constant in the MySQL world that available QA tools often leave something to be desired.  The randgen is a literal wonder-tool for database testing, but it is also occasionally frustrating / doesn’t scratch every testing itch.  It is based on technology SQL Server was using in 1998 (MySQL began using it in ~2007, IIRC).  So this is no knock, it is merely meant to be an example of a poor QA engineer’s frustrations ; )  While the current Xtrabackup test suite is commendable, it also has its limitations. Enter the flexible, adaptable, and expressive answer: dbqp.

One of my demos at the dev meeting was showing how we can set up tests for Xtrabackup using the unittest paradigm.  While this sounds fancy, basically, we take advantage of Python’s unittest and write classes that use their code.  The biggest bit dbqp does is search the specified server code (to make sure we have everything we should), allocate and manage servers as requested by the test cases, and do some reporting and management of the test cases.  As the tool matures, I will be striving to let more of the work be done by unittest code rather than things I have written : )

To return to my main point, we now have two basic tests of xtrabackup:

Basic test of backup + restore:
  1. Populate server
  2. Take a validation snapshot (mysqldump)
  3. Take the backup (via innobackupex)
  4. Clean datadir
  5. Restore from backup
  6. Take restored state snapshot and compare to original state
Slave setup
  1. Similar to our basic test except we create a slave from the backup, replicating from the backed up server.
  2. After the initial setup, we ensure replication is set up ok, then we do additional work on the master and compare master and slave states

One of the great things about this is that we have the magic of assertions.  We can insert them at any point of the test we feel like validating and the test will fail with useful output at that stage.  The backup didn’t take correctly?  No point going through any other steps — FAIL! : )  The assertion methods just make it easy to express what behavior we are looking for.  We want the innobackupex prepare call to run without error?
Boom goes the dynamite!:

# prepare our backup
cmd = ("%s --apply-log --no-timestamp --use-memory=500M "
"--ibbackup=%s %s" %( innobackupex
, xtrabackup
, backup_path))
retcode, output = execute_cmd(cmd, output_path, exec_path, True)
self.assertEqual(retcode, 0, msg = output)

From these basic tests, it will be easy to craft more complex test cases.  Creating the slave test was simply matter of adapting the initial basic test case slightly.  Our plans include: *heavy* crash testing of both xtrabackup and the server, enhancing / expanding replication tests by creating heavy randgen loads against the master during backup and slave setup, and other assorted crimes against database software.  We will also be porting the existing test suite to use dbqp entirely…who knows, we may even start working on Windows one day ; )

These tests are by no means the be-all-end-all, but I think they do represent an interesting step forward.  We can now write actual, honest-to-goodness Python code to test the server.  On top of that, we can make use of the included unittest module to give us all sorts of assertive goodness to express what we are looking for.  We will need to and plan to refine things as time moves forward, but at the moment, we are able to do some cool testing tricks that weren’t easily do-able before.

If you’d like to try these tests out, you will need the following:
* dbqp (bzr branch lp:dbqp)
* DBD:mysql installed (test tests use the randgen and this is required…hey, it is a WONDER-tool!) : )
* Innobackupex, a MySQL / Percona server and the appropriate xtrabackup binary.

To run them:
$./dbqp.py –suite=xtrabackup_basic –basedir=/path/to/mysql –xtrabackup-path=/mah/path –innobackupex-path=/mah/other/path –default-server-type=mysql –no-shm

Some next steps for dbqp include:
1)  Improved docs
2)  Merging into the Percona Server trees
3)  Setting up test jobs in Jenkins (crashme / sqlbench / randgen)
4)  Other assorted awesomeness

Naturally, this testing goodness will also find its way into Drizzle (which currently has a 7.1 beta out).  We definitely need to see some Xtrabackup test cases for Drizzle’s version of the tool (mwa ha ha!) >: )

Categories: Libdrizzle News

Mark Atwood: Drizzle: Seeking tiny contributions, leading to big things

Libdrizzle News - Mon, 11/07/2011 - 17:24

The Drizzle project regularly gets people asking what they can do to get involved in the project.

One very easy way to brush up on your C++ skills and dip your toe into our open development process is to fix minor warnings.

We are very proud that Drizzle builds with zero warnings for with "gcc -Wall -Wextra".

But we can be even better!  Our JenkinsCI system has a target that is even more picky, and also a target that runs cppcheck.

Go to one of those pages, pick a build log off the build history, find a warning that you think you can fix, and then ask us in the #drizzle channel on Freenode how to send your fix to us.

After you've done that a few times, you'll be ready to fix some low hanging fruit.

We've had people graduate from this process into becoming a Google Summer of Code student, and eventually having a full time paying job hacking on Drizzle and other open source software.

And it all starts with writing a simple warning fix.

Categories: Libdrizzle News

Henrik Ingo: Slides for Fixed in Drizzle talk, Percona Live UK 2011

Libdrizzle News - Sat, 11/05/2011 - 11:54

Here are the slides to my second talk at last week's Percona Live event in London:

Fixed in drizzle
View more presentations from Henrik Ingo

read more

Categories: Libdrizzle News

Official Drizzle Blog: Fremont beta (2011.10.28) has been released

Libdrizzle News - Tue, 10/25/2011 - 21:43

It is finally here!

The Fremont beta is out and ready to be tested.

In this release:

The Drizzle download file can be found here

Categories: Libdrizzle News

DrizzleDB: #perconalive is just around the corner. @h_ingo Henrik Ingo will be talking about #Drizzle and all the "gotchas" that have been removed.

@DrizzleDB on Twitter - Wed, 10/19/2011 - 23:45
DrizzleDB: #perconalive is just around the corner. @h_ingo Henrik Ingo will be talking about #Drizzle and all the "gotchas" that have been removed.

DrizzleDB: RT @vjsamuel1990: @DrizzleDB is currently a featured project on @launchpad_net

@DrizzleDB on Twitter - Wed, 10/19/2011 - 23:41
DrizzleDB: RT @vjsamuel1990: @DrizzleDB is currently a featured project on @launchpad_net

DrizzleDB: @DrizzleDB is currently a featured project on @launchpad_net

@DrizzleDB on Twitter - Mon, 10/17/2011 - 16:57
DrizzleDB: @DrizzleDB is currently a featured project on @launchpad_net

Patrick Crews: Drizzle / dbqp updates

Libdrizzle News - Fri, 10/14/2011 - 20:31

Just wanted to blog about some of the latest updates to dbqp.  We just merged some interesting changes into Drizzle (just in time for the impending Fremont beta).  In additional to general code cleanup / reorganization, we have the following goodies:

Randgen in the Drizzle tree

One of the biggest things is that the random query generator (aka randgen) is now part of the Drizzle tree.  While I did some of the work here, the major drivers of this happening were Brian and Stewart:

  1. Brian makes a fair argument that the easier / more convenient it is to run a test, the greater the likelihood of it being run.  Additional tools to install, etc = not so much.  Having something right there and ready to go = win!
  2. Stewart is also a fan of convenience, lotsa testing, and working smarter, not harder.  As a result, he did the initial legwork on merging the randgen.  I do suspect there is still much for me to learn about properly bzr joining trees and whatnot, but we’ll get it right soon enough ; )

This doesn’t mean we won’t be contributing any changes we make back to the main randgen project / branch, it is strictly to facilitate more testing for Drizzle.  As we already have our randgen tests packaged into dbqp-runnable suites, running these tests is even easier : )

–libeatmydata

Another request fulfilled in this update is the ability to use Stewart’s libeatmydata to speed up testing.  By default, dbqp uses shared memory as a workdir, similar to mysql-test-run’s –mem option (this can be bypassed in dbqp with –no-shm, fyi).  However, this isn’t always perfect or desirable to do.

An alternative is to use libeatmydata, which disables fsync() calls.  As the name implies, you don’t want to use it if care about your data, but for general testing purposes, it can greatly speed up test execution.

If you have the library installed / on your machine, you can use it like so:  ./dbqp –libeatmydata [--libeatmydata-path ] …

By default, libeatmydata-path is /usr/local/lib/libeatmydata.so (as if you used make install)

Multiple server types

IMHO, this is one of the coolest new tricks.  dbqp can now handle more than just Drizzle servers / source!  The ultimate idea is to allow tests that utilize more than one type / version of a server to have more interesting tests : )  This will be useful for scenarios like testing Drizzledump migration as we can feed in one (or more) MySQL servers and a Drizzle tree and make sure we can migrate data from all of them.

We also intend to utilize dbqp for testing a variety of Percona products, and it is kind of handy to be able to run the code you are testing ; )  I already have the tool running Percona / MySQL servers and have some randgen tests working:


$ ./dbqp.py --default_server_type=mysql --basedir=/percona-server/Percona-Server --mode=randgen
Setting --no-secure-file-priv=True for randgen usage...
20111013-163443 INFO Linking workdir /dbqp/workdir to /dev/shm/dbqp_workdir_pcrews_9dbc7e8a-2872-45a9-8a07-f347f6184246
20111013-163443 INFO Using mysql source tree:
20111013-163443 INFO basedir: /percona-server/Percona-Server
20111013-163443 INFO clientbindir: /percona-server/Percona-Server/client
20111013-163443 INFO testdir: /dbqp
20111013-163443 INFO server_version: 5.5.16-rel21.0
20111013-163443 INFO server_compile_os: Linux
20111013-163443 INFO server_platform: x86_64
20111013-163443 INFO server_comment: (Percona Server with XtraDB (GPL), Release rel21.0, Revision 188)
20111013-163443 INFO Using default-storage-engine: innodb
20111013-163443 INFO Using testing mode: randgen
20111013-163443 INFO Processing test suites...
20111013-163443 INFO Found 5 test(s) for execution
20111013-163443 INFO Creating 1 bot(s)
20111013-163449 INFO Taking clean db snapshot...
20111013-163452 INFO bot0 server:
20111013-163452 INFO NAME: s0
20111013-163452 INFO MASTER_PORT: 9307
20111013-163452 INFO SOCKET_FILE: /dbqp/workdir/bot0/s0/var/s0.sock
20111013-163452 INFO VARDIR: /dbqp/workdir/bot0/s0/var
20111013-163452 INFO STATUS: 1
20111013-163506 ===============================================================
20111013-163506 TEST NAME [ RESULT ] TIME (ms)
20111013-163506 ===============================================================
20111013-163506 main.blob [ pass ] 8624
20111013-163516 main.create_drop [ pass ] 2862
20111013-163524 main.many_indexes [ pass ] 1429
20111013-163547 main.optimizer_subquery [ pass ] 17153
20111013-163558 main.outer_join [ pass ] 4243
20111013-163558 ===============================================================
20111013-163558 INFO Test execution complete in 69 seconds
20111013-163558 INFO Summary report:
20111013-163558 INFO Executed 5/5 test cases, 100.00 percent
20111013-163558 INFO STATUS: PASS, 5/5 test cases, 100.00 percent executed
20111013-163558 INFO Spent 34 / 69 seconds on: TEST(s)
20111013-163558 INFO Test execution complete
20111013-163558 INFO Stopping all running servers...

Expect to see this up and running tests against Percona Server in the next week or so.  I’ll be writing more about this soon.

Native / unittest mode

This hasn’t made it into the Drizzle tree yet.  To ease merging the code with Percona Server / Xtrabackup, I’ve created a separate launchpad project.  One of the things we needed was the ability to write complex tests directly.  It is currently easy to plug new tools into dbqp, but we essentially needed a new tool for certain testing needs.

Our solution for this was to allow dbqp to run python unittest modules.  We still have a bit of work to do before we have some demo tests ready, but we will be creating some expanded Xtrabackup tests using this system very soon.  So far, it is turning out to be pretty neat:


./dbqp.py --default_server_type=mysql --basedir=/percona-server/Percona-Server --mode=native
20111013-190744 INFO Killing pid 1747 from /dbqp/workdir/bot0/s0/var/run/s0.pid
20111013-190744 INFO Linking workdir /dbqp/workdir to /dev/shm/dbqp_workdir_pcrews_9dbc7e8a-2872-45a9-8a07-f347f6184246
20111013-190744 INFO Using mysql source tree:
20111013-190744 INFO basedir: /percona-server/Percona-Server
20111013-190744 INFO clientbindir: /percona-server/Percona-Server/client
20111013-190744 INFO testdir: /dbqp
20111013-190744 INFO server_version: 5.5.16-rel21.0
20111013-190744 INFO server_compile_os: Linux
20111013-190744 INFO server_platform: x86_64
20111013-190744 INFO server_comment: (Percona Server with XtraDB (GPL), Release rel21.0, Revision 188)
20111013-190744 INFO Using default-storage-engine: innodb
20111013-190744 INFO Using testing mode: native
20111013-190744 INFO Processing test suites...
20111013-190744 INFO Found 1 test(s) for execution
20111013-190744 INFO Creating 1 bot(s)
20111013-190749 INFO Taking clean db snapshot...
20111013-190750 INFO bot0 server:
20111013-190750 INFO NAME: s0
20111013-190750 INFO MASTER_PORT: 9306
20111013-190750 INFO SOCKET_FILE: /dbqp/workdir/bot0/s0/var/s0.sock
20111013-190750 INFO VARDIR: /dbqp/workdir/bot0/s0/var
20111013-190750 INFO STATUS: 1
20111013-190756 ===============================================================
20111013-190756 TEST NAME [ RESULT ] TIME (ms)
20111013-190756 ===============================================================
20111013-190756 main.example_test [ pass ] 1
20111013-190756 test_choice (example_test.TestSequenceFunctions) ... ok
20111013-190756 test_sample (example_test.TestSequenceFunctions) ... ok
20111013-190756 test_shuffle (example_test.TestSequenceFunctions) ... ok
20111013-190756
20111013-190756 ----------------------------------------------------------------------
20111013-190756 Ran 3 tests in 0.000s
20111013-190756
20111013-190756 OK
20111013-190756
20111013-190756 ===============================================================
20111013-190756 INFO Test execution complete in 6 seconds
20111013-190756 INFO Summary report:
20111013-190756 INFO Executed 1/1 test cases, 100.00 percent
20111013-190756 INFO STATUS: PASS, 1/1 test cases, 100.00 percent executed
20111013-190756 INFO Spent 0 / 6 seconds on: TEST(s)
20111013-190756 INFO Test execution complete
20111013-190756 INFO Stopping all running servers...

This really only scratches the surface of what can happen, but I’ll be writing more in-depth articles on what kind of tricks we can pull off as the code gets more polished.

Three non-testing bits:

1)  Percona Live London is just around the corner and members of the Drizzle team will be there.

2)  We are *this* close to Fremont beta being ready.  The contributions and feedback have been most welcome.  Any additional testing / etc are most appreciated.

3)  Drizzle is now part of the SPI!

 

Categories: Libdrizzle News

Official Drizzle Blog: Drizzle source tarball 2011.10.27 has been released

Libdrizzle News - Tue, 10/11/2011 - 22:49

Drizzle source tarball, version 2011.10.27 has been released.

In this release:

  • Continued code refactoring
  • Cleanup of test system
  • Document cleanup

 

The Drizzle download file can be found here

Categories: Libdrizzle News

DrizzleDB: Henrik Ingo (@h_ingo) speaking on Drizzle's Progress at Percona Live London October 25th. Discount Code DrizzlePLUK. http://t.co/0lSbxnTN

@DrizzleDB on Twitter - Thu, 10/06/2011 - 21:34
DrizzleDB: Henrik Ingo (@h_ingo) speaking on Drizzle's Progress at Percona Live London October 25th. Discount Code DrizzlePLUK. http://t.co/0lSbxnTN

DrizzleDB: Correction: Stewart Smith speaking on Drizzle at Percona Live London October 25th. Discount code "DrizzlePLUK" http://t.co/G8MDx7Dx

@DrizzleDB on Twitter - Thu, 10/06/2011 - 21:29
DrizzleDB: Correction: Stewart Smith speaking on Drizzle at Percona Live London October 25th. Discount code "DrizzlePLUK" http://t.co/G8MDx7Dx

DrizzleDB: Stewart Smith speaking about the GA release of Drizzle at Percona Live London October 24th. Discount code "DrizzlePLUK" http://t.co/G8MDx7Dx

@DrizzleDB on Twitter - Thu, 10/06/2011 - 21:25
DrizzleDB: Stewart Smith speaking about the GA release of Drizzle at Percona Live London October 24th. Discount code "DrizzlePLUK" http://t.co/G8MDx7Dx

Vijay Samuel: vjsamuel

Libdrizzle News - Thu, 10/06/2011 - 15:58

Its been a while since I ve blogged but I couldn’t think of a better time to resume blogging than when Drizzle was officially became associated to Software in the Public Interest. I ve been a part of Drizzle for almost a year and a half now and my passion for Drizzle seems to grow every day. It is always good to see changes that happen for good and this is one of them I guess. Now that Drizzle is a part of SPI, it has a legal entity behind it which is always good. How can you benefit from this you may ask. If you are a US tax payer, then any donation that you make will be tax deductible and all your valuable contributions will be used towards the betterment of Drizzle. The easiest way to donate is using a credit card at Click & Pledge. The SPI website lists some alternative methods such as using a cheque.  So, please do make your valuable contributions towards Drizzle. As always I feel proud to be a part of the Drizzle family and will continue to strive for the betterment of Drizzle.


Categories: Libdrizzle News

Official Drizzle Blog: Drizzle joins Software in the Public Interest, can take (tax-deductible) donations

Libdrizzle News - Thu, 10/06/2011 - 13:36

Since it's inception in 2008 Drizzle has taken the approach of an open
source community project. It's been run as a meritocracy by its
developer community, with developers from various companies and just
individuals. Even though at first Sun and then Rackspace did sponsor a
core team to work full time on the project, the project is not owned
by any particular company, but Drizzle is developed by a vibrant and
diverse community. We've been proud to have between 20-40 active
contributors each month.

With the first stable release in March 2011, interest in the project
grew. At this point we also decided it was time to solidify and
clarify the status of the project as a non-profit community project.
Out of a couple options available, we chose to become an Associated
Project at the Software in the Public
Interest
. The SPI is a charitable US non-profit corporations that
acts as an umbrella
organization to many open source projects
, including some well
known ones like Debian and PostgreSQL. Drizzle was accepted as an SPI
Associated Project by the Board on August
10th
.

At this point, we would like to extend our gratitude to Josh Berkus (a
PostgreSQL lead developer) who guided and sponsored Drizzle through
the application process! We appreciate not just your experience and
expertise with the SPI, but also the gesture of friendly help between
two open source database projects.

Having a legal entity behind Drizzle has a number of useful benefits
and any open source project that is bigger than just one or two guys
should seriously consider using one of these umbrella organizations.

One benefit that is available to us starting this week is that you can
now donate money to Drizzle via the SPI. The easiest way to donate is
using a credit card at Click & Pledge. The SPI website lists some alternative methods such as
using a cheque. For US tax payers donations are tax-deductible and if
you are a business you can of course write the donation off as an
expense.

Donations made via the SPI bank account will be properly accounted for
by the SPI treasurer. Please read above links on how to "earmark" your
donation for Drizzle. Available funds will be used by the Drizzle
project primary for expenses such as legal or IT infrastructure and
secondary for arranging developer meetings and if possible sponsoring
attendance at conferences. Note that if you want to sponsor
development of Drizzle, such as a particular feature, and you have a
large enough budget to be usable for that purpose, we recommend you
rather contract such development directly via one of the
commercial service providers that employ Drizzle developers
.

Categories: Libdrizzle News

Patrick Crews: Drizzle multi-master testing!

Libdrizzle News - Tue, 10/04/2011 - 21:19

So, it has been a while since I’ve blogged.  As some of you may have read, I have a new job and Stewart and I have been busy planning all kinds of testing goodness for Percona >: ) (I’ve also been recovering from trying to keep up with Stewart!)

Rest assured, gentle readers, that I have not forgotten everyone’s favorite modular, community-driven database ; )  Not by a long-shot.  I have some major improvements to dbqp getting ready for a merge (think randgen in-tree / additional testing modes / multiple basedirs of multiple types).  Additionally, I’ve been cooking up some code to test the mighty Mr. Shrews’ multi-master code (mwa ha ha!)

What I’ve done is allow for a new option to be used with a test’s .cnf file (this is a dbqp thing, won’t work with standard drizzle-test-run).  If the runner sees this request, it will generate a multi-master config file from the specified servers’ individual slave.cnf files. 

Here is a sample config:

[test_servers]
servers = [[--innodb.replication-log],[--innodb.replication-log],[--plugin-add=slave --slave.config-file=$MASTER_SERVER_SLAVE_CONFIG]]

[s2]
# we tell the system that we want
# to generate a multi-master cnf file
# for the 3rd server to use, that
# has the first two servers as masters
# the final file is written to the first
# server's general slave.cnf file
gen_multi_master_cnf= 0,1

A good rundown of the file’s contents can be found on Shrews’ blog here, but the end result looks like this:

ignore-errors

[master1]
master-host=127.0.0.1
master-port=9306
master-user=root
master-pass=''

[master2]
master-host=127.0.0.1
master-port=9312
master-user=root
master-pass=''

I tried cooking up a basic test case where we spin up 3 servers – 2 masters and one slave.  One master 1, we create table t1:


CREATE TABLE t1 (a int not null auto_increment, primary key(a));

On master 2, table t2:


CREATE TABLE t2 (a int not null auto_increment, primary key(a));

We insert some records into both tables, then check that our slave has everything! Sounds simple, right?

Sigh. If only. It seems that we are running into some issues when we try to record the test – you can read the bug here

We see some interesting output in the slave’s logs before it crashes:

$ cat workdir/bot0/s2/var/log/s2.err
InnoDB: Doublewrite buffer not found: creating new
InnoDB: Doublewrite buffer created
InnoDB: 127 rollback segment(s) active.
InnoDB: Creating foreign key constraint system tables
InnoDB: Foreign key constraint system tables created
(SQLSTATE 00000) Duplicate entry '772-1' for key 'PRIMARY'
Failure while executing:
INSERT INTO `sys_replication`.`queue` (`master_id`, `trx_id`, `seg_id`, `commit_order`, `originating_server_uuid`, `originating_commit_id`, `msg`) VALUES (2, 772, 1, 1, 'ac9c8ac0-8f10-474b-9bbd-b61d2cdb2b93' , 1, 'transaction_context {
server_id: 1
transaction_id: 772
start_timestamp: 1317760732106016
end_timestamp: 1317760732106017
}
event {
type: STARTUP
}
segment_id: 1
end_segment: true
')

Replication slave: Unable to insert into queue.
Replication slave: drizzle_state_read:lost connection to server (EOF)
Lost connection to master. Reconnecting.
Replication slave: drizzle_state_connect:could not connect
111004 16:39:05 InnoDB: Starting shutdown...

Additionally, you can just try the setup with –start-and-exit:

$ ./dbqp --suite=slave --start-and-exit multi_master_basic
20111004-170033 INFO Using Drizzle source tree:

20111004-170033 INFO Taking clean db snapshot...
20111004-170033 INFO Taking clean db snapshot...
20111004-170033 INFO Taking clean db snapshot...
20111004-170035 INFO bot0 server:
20111004-170035 INFO NAME: s0
20111004-170035 INFO MASTER_PORT: 9306
20111004-170035 INFO DRIZZLE_TCP_PORT: 9307
20111004-170035 INFO MC_PORT: 9308
20111004-170035 INFO PBMS_PORT: 9309
20111004-170035 INFO RABBITMQ_NODE_PORT: 9310
20111004-170035 INFO VARDIR: /drizzle_mm_test/tests/workdir/bot0/s0/var
20111004-170035 INFO STATUS: 1
20111004-170035 INFO bot0 server:
20111004-170035 INFO NAME: s1
20111004-170035 INFO MASTER_PORT: 9312
20111004-170035 INFO DRIZZLE_TCP_PORT: 9313
20111004-170035 INFO MC_PORT: 9314
20111004-170035 INFO PBMS_PORT: 9315
20111004-170035 INFO RABBITMQ_NODE_PORT: 9316
20111004-170035 INFO VARDIR: /drizzle_mm_test/tests/workdir/bot0/s1/var
20111004-170035 INFO STATUS: 1
20111004-170035 INFO bot0 server:
20111004-170035 INFO NAME: s2
20111004-170035 INFO MASTER_PORT: 9318
20111004-170035 INFO DRIZZLE_TCP_PORT: 9319
20111004-170035 INFO MC_PORT: 9320
20111004-170035 INFO PBMS_PORT: 9321
20111004-170035 INFO RABBITMQ_NODE_PORT: 9322
20111004-170035 INFO VARDIR: /drizzle_mm_test/tests/workdir/bot0/s2/var
20111004-170035 INFO STATUS: 1
20111004-170035 INFO User specified --start-and-exit. dbqp.py exiting and leaving servers running...
pcrews@mister:/drizzle_mm_test/tests$ ps -al
F S UID PID PPID C PRI NI ADDR SZ WCHAN TTY TIME CMD
0 S 1000 18652 1 2 80 0 - 112094 poll_s pts/2 00:00:00 lt-drizzled
0 S 1000 18688 1 3 80 0 - 112096 poll_s pts/2 00:00:00 lt-drizzled
0 S 1000 18721 1 3 80 0 - 156326 poll_s pts/2 00:00:00 lt-drizzled
0 R 1000 18780 15985 0 80 0 - 3375 - pts/2 00:00:00 ps
0 S 1000 32463 30047 0 80 0 - 11272 poll_s pts/1 00:00:01 ssh

From here, we can connect to the slave and check out sys_replication.applier_state:

$ drizzle -uroot -p9318 test
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the Drizzle client.. Commands end with ; or \g.
Your Drizzle connection id is 216
Connection protocol: mysql
Server version: 2011.09.26.2427 Source distribution (drizzle_mm_test)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

drizzle> use sys_replication;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Schema changed
drizzle> show tables;
+---------------------------+
| Tables_in_sys_replication |
+---------------------------+
| applier_state |
| io_state |
| queue |
+---------------------------+
3 rows in set (0.001641 sec)

drizzle> select * from applier_state;
+-----------+------------------------+--------------------------------------+-----------------------+---------+-----------+
| master_id | last_applied_commit_id | originating_server_uuid | originating_commit_id | status | error_msg |
+-----------+------------------------+--------------------------------------+-----------------------+---------+-----------+
| 1 | 0 | f716781f-8c00-4b81-82c6-62039136d616 | 0 | RUNNING | |
| 2 | 3 | df7f2f6e-dba4-43ea-a674-fa4a3709865b | 3 | RUNNING | |
+-----------+------------------------+--------------------------------------+-----------------------+---------+-----------+
2 rows in set (0.000928 sec)

drizzle> select * from io_state;
+-----------+---------+-----------+
| master_id | status | error_msg |
+-----------+---------+-----------+
| 1 | STOPPED | |
| 2 | RUNNING | |
+-----------+---------+-----------+
2 rows in set (0.000839 sec)

drizzle>

So, it looks like the slave knows about both masters, but for some reason, the applier from master 1 is stopped : (
At any rate, there is a bug open on this and it could be something in my config(?) It’s been a while since I’ve played with replication and I know there has been some tinkering under the hood since then : )

The branch with the test code can be found here:
lp:~patrick-crews/drizzle/dbqp_multi_master_test

At the very least, we can now create tests that use this feature, which will help ensure that it stays on the path of solid code in the future! How about anyone out there? Has anyone been using multi-master? If so, can you share any setups / tests? Extra information would be most appreciated : )

Categories: Libdrizzle News

MySQL Performance Blog: Percona Welcomes Patrick Crews

Libdrizzle News - Mon, 10/03/2011 - 23:13

I am very happy to welcome Patrick Crews to the Percona development team. Patrick joins Percona at a very exciting time for the development team. We are getting regular releases of Percona Server and Percona Xtrabackup out the door, we have been heavily using the Jenkins continuous integration system to maintain and improve the quality of the products we ship and we just upgraded our documentation publishing platform for both Percona Server (5.1 and 5.5) and Percona Xtrabackup.

We are at the natural point to expand our QA efforts – and that’s where Patrick joins us.

Patrick has been doing QA in the MySQL world for a while now, and has extensive experience with both MySQL and Drizzle. His work has included use of a variety of testing tools such as the randgen (random query generator) project to which he contributes.

As a Drizzle developer, he saw the code get to its first GA release. This included testing a completely rewritten replication system, drizzledump’s evolution to a migration tool, as well as creating a new pluggable testing system for the project (dbqp – expect to hear a lot more on this in the months to come).

Patrick’s role will have him working on both Percona Server, XtraDB, and XtraBackup. This will include creating more advanced tests and test systems for our development needs which will naturally also improve the testing of Drizzle due to sharing of common code.

Patrick has a blog over at www.wc220.com where he writes about Drizzle QA and other topics.

Categories: Libdrizzle News

MySQL Performance Blog: Updated Percona Server and Percona XtraBackup documentation

Libdrizzle News - Mon, 10/03/2011 - 07:07

We’ve just gone live with a new way of publishing and maintaining documentation for Percona Server and Percona XtraBackup. We are now using Sphinx to generate the documentation that we publish on the web site. Sphinx was originally created for the new Python documentation, and has won us over due to the simple markup, ease of cross-referencing, ability to keep our documentation source along with our source code and support for multiple output formats.

This moves our documentation workflow to be the same as for source code. A developer (or writer) will create a bzr branch on launchpad and submit a merge request. This means documentation review now goes through the same system as code review.

We have also set up a Jenkins job to automatically update the documentation on the percona.com web site when changes are made to the source repositories. This means that bug fixes and improvements to the documentation can transparently and automatically be published with a minimal amount of turn-around.

An added benefit for XtraBackup is that it it will enable us to easily share documentation for the Drizzle version of XtraBackup as Drizzle uses Sphinx for its documentation too. The new setup for Percona documentation was based on that of the Drizzle project.

The end result? Better and more frequently updated documentation.

Categories: Libdrizzle News

Stewart Smith: Speaking at Percona Live London 2011 (on Drizzle!)

Libdrizzle News - Sun, 10/02/2011 - 23:15

Both Henrik and myself will be at Percona Live London 2011 in late October speaking on the wonderful Drizzle database server.

Other speakers at the conference will be talking about a wide range of topics surrounding the MySQL ecosystem including performance monitoring, backup, search, scaling and data recovery.

P.S. I do have a discount code – ask me in the comments for it!

Categories: Libdrizzle News

MySQL Performance Blog: Speaking at HighLoad conference, Moscow, Russia, Training and Hiring

Libdrizzle News - Fri, 09/30/2011 - 14:27

I’m going to be speaking on Highload++ conference October 3,4 in Moscow, Russia. This is a great conference which gathers amazing quality of speakers from Russia and around the world and I usually learn a lot and enjoy talking to a lot of great people on this event.

My talk is going to be about new developments in MySQL Server 5.5, 5.6, Percona Server, MariaDB and Drizzle as it relates to high volume/large scale projects. There is a lot of really cool things happening in MySQL space recently and I would love to share those with you.

I’m also doing a training session 5th of October, which will be in depth training session based on Percona’s Training for MySQL Developers. In fact this is a great learning experience both for MySQL Developers and DBAs and this is unique opportunity to attend this course
given in Russian. Bring your laptop if you have one for more Hands On experience.

If you would like to do business with Percona I will be available for business development and consulting meetings on Thursday and Friday next week.

Finally note We’re Hiring worldwide for most of positions, so if you’re interested joining Percona Team, come and talk to me. We have both technical and non technical positions, in particular we’re looking for Sales and Business Development person in Russia.

Why so late announcement. Ah if you must know I did not know if I will be able to make it until very last minute. My passport was stalled in the British Consulate getting visa to UK for Percona Live London in late October. It is great we could get the documents back in time for my travel.

Categories: Libdrizzle News

Pages