Skip directly to content

Libdrizzle News

Subscribe to Libdrizzle News feed
Updated: 5 min 8 sec ago

Pythian Group: MySQL Conference 2012 – The Keynotes (3)

Wed, 04/11/2012 - 16:57
And lastly, from none other than The Brian Aker, a keynote on The New MySQL Cloud Ecosystem. He was formerly the Director of Architecture for MySQL and also the creator of Drizzle. He is currently a fellow at HP, leading their cloud architecture group. A little history of MySQL of course. The drivers as seen [...]
Categories: Libdrizzle News

Brian Aker: Drizzle 7.1 Released

Tue, 04/10/2012 - 22:55
Drizzle 7.1 was released!

The laundry list for features and improvements are pretty long, so I will leave that to the main blog announcement.

The part that I find the most interesting? 7.1 was the effort of a lot of developers and people who do the heavy lifting of promoting the project (and the efforts around a project like this are huge). It takes the resources of a half a dozen companies directly, and a number of companies indirectly.

Compared to 7.0, Drizzle was done by volunteers, not paid Drizzle employees, who find the project fun and interesting. These are people who want to be involved with the Open Source Community, and who find value in the work (and looking at our incoming Google Summer of Code applicants there are more people to come).

The biggest surprise that some might fine? Much of the 7.1 effort was done by new developers. These are people who never worked at MySQL, Sun, or Oracle.

Contributions by new individuals make up the bulk of the 7.1 effort, and I am very happy to see that. It it is evidence of the ongoing growth of the Drizzle community.

Categories: Libdrizzle News

MySQL Performance Blog: Percona welcomes Drizzle 7.1

Tue, 04/10/2012 - 16:11

The Drizzle team has published the first Drizzle 7.1 stable release. The announcement on the Drizzle blog covers some of the major improvements in this release over the previous stable release, Drizzle 7.0. I plan to write a variety of blog posts on each of the new features over the coming weeks. Major features include:

  • Multi sourced replication
  • XtraBackup included
  • IPv6 data type
  • ZeroMQ plugin
  • Log output to syslog is enabled by default
  • Updated DEB and RPM packages
  • Query log plugin and auth_schema
  • JS plugin: execute JavaScript code as a Drizzle function
  • HTTP JSON API (experimental)
  • Some of the Percona InnoDB patches have been merged

Since Drizzle 7.0 (which was released last year), there have been over 3,500 commits to Drizzle from around 50 contributors (no doubt the translations add more too).

Some interesting statistics about this Drizzle release:

  • The diffstat from the previous stable release is: 3891 files changed, 473733 insertions(+), 210435 deletions(-)
  • The size of the Drizzle kernel (drizzled directory in the source) has shrunk from 149,586 to 146,454 lines of code (according to sloccount).
  • Out of 78 plugins there are only ten that have more than 1,000 lines of code. The remaining 68 plugins are not all trivial either, they include: authentication via PAM, authentication via HTTP, various DATA_DICTIONARY tables, the Drizzle protocol plugin, authentication from a file, MySQL UNIX socket protocol, javascript plugin, authentication via tables (similar to MySQL), RabbitMQ, regex policy, logging to syslog, authentication via LDAP. This shows that major pieces of functionality can be implemented in relatively small amounts of code.

Installing Drizzle 7.1.33-stable

The source tar package is available from the Drizzle Launchpad download area. Please see the Drizzle manual on how to install Drizzle 7.1.33-stable from source tar package.

The Ubuntu packages are available from the Drizzle PPA. Instructions on how to install from the PPA are in the Drizzle manual.

CentOS 6 packages are available from download.drizzle.orgInstructions on how to install them with yum are in the Drizzle manual.

Note: For Drizzle 7.1.33-stable, only 64-bit packages were built for RedHat/Centos.

Categories: Libdrizzle News

Daniel Nichter: Drizzle 7.1 released

Fri, 04/06/2012 - 23:57
Drizzle 7.1 has been released.  In my frank opinion, there’s no longer any reason to use 7.0; everyone should upgrade to 7.1 because it is far superior.  Read the release announce for the list of new and fixed things.  Most importantly, imho: sweeping updates to docs.drizzle.org.  This is also good news because in a few [...]
Categories: Libdrizzle News

Vijay Samuel: vjsamuel

Wed, 04/04/2012 - 16:21

The GA of Drizzle is available for public use. You can see our blog post for the new improvements available with this release. I’ve been with Drizzle for two years now and it is good to see Drizzle mature. This release has given more importance to the users as we’ve done a lot of work on improving documentation and fixing a lot of bugs. We’ve also moved forward in the quest of bringing true multi tenancy.  The release is right on time for Percona Live MySQL Conference which is scheduled for next week. This release is even more special to me as I have the privilege of being Drizzle’s Release Manager and had the opportunity to announce to the world the GA of Drizzle 7.1. As always I’m proud to be a Drizzle dev


Categories: Libdrizzle News

Official Drizzle Blog: Drizzle 7.1.33-stable has been released

Wed, 04/04/2012 - 15:25

The Global Drizzle Development Team is pleased to announce the immediate availability of Drizzle 7.1.33-stable. The first stable release of Drizzle 7.1 and the result of 12 months of hard work from contributors around the world.

Improvements in Drizzle 7.1 compared to 7.0

- Xtrabackup is included (in-tree) by Stewart Smith

- Multi-source replication by David Shrewsbury

- Improved execute parser by Brian Aker and Vijay Samuel - Servers are identified with UUID in replication by Joe Daly

- HTTP JSON API (experimental) by Stewart Smith

- Percona Innodb patches merged by Laurynas Biveinis

- JS plugin: execute JavaScript code as a Drizzle function by Henrik Ingo

- IPV6 data type by Muhammad Umair

- Improvements to libdrizzle client library by Andrew Hutchings and Brian Aker

- Query log plugin and auth_schema by Daniel Nichter

- ZeroMQ plugin by Markus Eriksson

- Ability to publish transactions to zeromq and rabbitmq by Marcus Eriksson

- Replication Dictionary by Brian Aker

- Log output to syslog is enabled by default by Brian Aker

- Improvements to logging stats plugin

- Removal of drizzleadmin utility (you can now do all administration from drizzle client itself) by Andrew Hutchings

- Improved Regex Plugin by Clint Byrum

- Improvements to pandora build by Monty Taylor

- New version numbering system and support for it in pandora-build by Henrik Ingo

- Updated DEB and RPM packages, by Henrik Ingo

- Revamped testing system Kewpie all-inclusive with suites of randgen, sysbench, sql-bench, and crashme tests by Patrick Crews

- Removal of HailDB engine by Stewart Smith

- Removal of PBMS engine

- Continued code refactoring by Olaf van der Spek, Brian Aker and others

- many bug fixes

Brian Aker ,Mark Atwood- Continuous Integration

Vijay Samuel - Release Manager

Installing Drizzle 7.1.33-stable

The source tar package is available from our Launchpad download area. Please see the manual on how to install Drizzle 7.1.33-stable from source tar package.

The Ubuntu packages are available from the Drizzle PPA. Instructions on how to install from the PPA are in the Drizzle manual.

CentOS 6 packages are available from download.drizzle.org. Instructions on how to install them with yum are in the Drizzle manual.

Note: For Drizzle 7.1.33-stable, only 64-bit packages were built for RedHat/Centos.

Known issues

All of the optional plugins are packaged in their own package. These are named drizzle-plugin-*. If a plugin is installed, it is also configured to load by default. (see /etc/drizzle/conf.d/pluginname.cnf) However, not all plugins actually work without further configuration. These plugins will cause the server startup to fail. The workaround is to either configure the plugin to work, or uninstall it if you don't want to use it. (Related blueprint) In short, you should only install the plugins you want to use. Don't install all available plugins. You also need to use "yum erase" and "apt-get purge" to uninstall plugins.

Please post bugs on the Drizzle bug tracker and questions and general feedback to drizzle-discuss mailing list.

Drizzle documentation

Much improved documentation is available at docs.drizzle.org

Planet Drizzle

You can follow our developer blogs, with more information about the features in Drizzle 7.1 at planetdrizzle.org

Percona Live MySQL Conference and Drizzle Day

The Drizzle team will be well represented at next week's Percona Live MySQL Conference and Expo and the Drizzle Day in the Santa Clara Convention Center. There are a couple talks from Drizzle developers and users in the main conference, and a full day on Friday, Apr 13. Drizzle Day is free to attend (though donations are encouraged). Please join us in Santa Clara and come and say hi!

 

Commercial 24/7 support is available

Several support vendors in the MySQL space provide commercial support and services for Drizzle 7.1. Click here for a list of support vendors that contributed to Drizzle during the 7.1 release cycle.

Info to press

Drizzle core developers are available for email and phone interviews

to discuss Drizzle 7.1 and also upcoming Drizzle topics. To arrange an

interview, please contact

In Americas: Brian Aker brian@tangent.org,  Mark Atwood me@mark.atwood.name, Kent Bozlinski : kent@bozlinski.com

In EMEA: Henrik Ingo henrik.ingo@avoinelama.fiIn APAC: Stewart Smith stewart.smith@percona.com

 

Categories: Libdrizzle News

MySQL Performance Blog: Diamond Keynote Panel, BOFs, Lightning Talks, and McAfee and AOL Sponsorships

Mon, 04/02/2012 - 17:01

I’m excited by all of the recent developments surrounding the Percona Live MySQL Conference and Expo! Our own Baron Schwartz will moderate the Diamond Keynote Panel entitled “Future Perfect: The Road Ahead for MySQL” which will feature a panel of MySQL industry leaders, including: Sundar Raghavan, director product management at Amazon; Paul Mikesell, CEO of Clustrix; a representative from HP; and, a representative from McAfee. The Diamond Sponsor Keynote Panel will take place at 9:30 a.m. on Thursday, April 12th and provide insight into the future of MySQL technology, adoption, and the ecosystem landscape. I am also very pleased to introduce two new sponsors including McAfee which recently joined as a Diamond Sponsor and AOL which recently agreed to participate as a lunch sponsor.

We also recently announced the Birds of a Feather Sessions which will take place on Tuesday at 6:30 p.m. and Wednesday at 9:00 p.m. BOFs enable attendees with the same project or topic interests to enjoy some quality face time. Topics include “Query Optimization”, “MySQL and Ruby on Rails”, “MySQL in the Cloud”, and much more.

We also published the list of Lightning Talks, a series of five-minute speaking opportunities during which attendees will propose, explain, exhort, and rant on a variety of MySQL-related topics. The one-hour Lightning Talks session will take place at 6:30 p.m. on Wednesday night during the Community Networking Reception. Topics include “The Five Minute Tour of Drizzle”, “How to Design a Scalable System”, and “How to Become a Rockstar DBA”.

The Percona Live MySQL Conference and Expo promises to be a great event! If you already registered, I look forward to seeing you in Santa Clara next week. If you haven’t registered but plan to attend, register now using discount code PerLiveSC and receive 10% off your registration fees. If you just cannot make it this year, keep your eyes open for recordings of the keynote sessions which we plan to post online following those events.

Categories: Libdrizzle News

Patrick Crews: Speaking at the Percona Live MySQL Conference and Expo

Fri, 03/30/2012 - 21:54

A number of people have already mentioned this, but the Percona Live MySQL Conference and Expo is just around the corner.
As Stewart has already blogged, there are a number of great sessions this year and I’m looking forward to several of them.

I’ll be giving a talk there as well -
It’s essentially all in the abstract, but I’ll be speaking about various functional testing tools that exist for MySQL-based systems.
Come to learn more about the random query generator, MTR, and kewpie and how they might be of use to you.

Speaking of kewpie, I’ll also be presenting about it at Drizzle Developer Day, which is on 4/13, the day after the main conference.
If you are interested in learning more about Drizzle, whether it be from hearing the various presentations to having a chance to chat with the developers and fellow enthusiasts, you should check it out.  Much like Stewart, I’m quite psyched about the event and doubly excited that my employer, Percona, is sponsoring it.

Additionally, there are other events that day, as Peter mentions here:

Categories: Libdrizzle News

Henrik Ingo: Simple GUI to edit JSON records in Drizzle

Thu, 03/29/2012 - 10:47

So yesterday I introduced the newly committed HTTP JSON key-value interface in Drizzle. The next step of course is to create some simple application that would use this to store data, this serves both as an example use case as well as for myself to get the feeling for whether this makes sense as a programming paradigm.

Personally, I have been a fan of the schemaless key-value approach ever since I graduated university and started doing projects with dozens of tables and hundreds of columns in total. Especially in small projects I always found the array structures in languages like PHP and Perl and Python to be very flexible to develop with. As I was developing and realized I need a new variable or new data field somewhere, it was straightforward to just toss a new key-value into the array and continue with writing code. No need to go back and edit some class definition. If I ever needed to find out what is available in some struct, I could always do dump_var($obj) to find out. Even large projects like Drupal get along with this model very well.

read more

Categories: Libdrizzle News

Henrik Ingo: Drizzle JSON HTTP interface now with key-value support

Mon, 03/26/2012 - 22:14

The thing I really like with open source is the feeling you get when people just show up from nowhere and do great things to some code you originally wrote. Thanks to this miracle, I can now also present to you version 0.2 of the Drizzle JSON HTTP support, featuring a "pure JSON key-value API" in addition to the original "SQL over HTTP" API in 0.1 version. Let's recap what happened:

  1. At Drizzle Day 2011, I proposed that Drizzle should make available a JSON NoSQL interface. Stewart took the bait and published json_server 0.1 a week later. This API still uses SQL, it's just that the client protocol is HTTP and JSON, into which the SQL is embedded. So I suppose it's not as sexy as real NoSQL, but pretty cool nevertheless.
  2. At the end of last Summer I had a lot of dead time so I started playing with Stewart's code to see what I could do. I added a new API in addition to Stewart's SQL-over-HTTP API that supports key-value operations in pure JSON, similar to what you see in CouchDB, MongoDB or Metabase. I got it working quite well, however, I never implemented a DELETE command, because I then drifted off to more important tasks, such as revamping the Drizzle version numbering and bringing DEB and RPM packaging up to date.
  3. Last week a new but very promising Drizzle hacker called Mohit came by, looking for things he could do. He had already fixed a simple low-hanging-bug and wanted something more. Since he was interested in the JSON API, I asked if he wants to finish the missing piece. With my helpful advice of "there is no documentation but if you look at the demo GUI you'll probably figure it out, then just look at the code for POST and implement DELETE instead". I was afraid that wasn't really helpful, but I was busy that day. No problem, the next day Mohit had pushed working DELETE implementation. The day after that he implemented the final missing piece, a CREATE TABLE functionality. I was both impressed and excited.

read more

Categories: Libdrizzle News

MySQL Performance Blog: Community events around Percona Live MySQL Conference and Expo 2012

Mon, 03/26/2012 - 19:45

We’re very pleased with community support for Percona Live MySQL Conference and Expo. We’ve got wonderful speakers providing content of phenomenal session lineup for the conference this year. We have community helping by spreading the world, about conference, picking talks and all king of other
wonderful contributions.

We also have series of events organized by Community and different organizations, taking place around the conference which you surely should not miss:

Pythian is stepping in to organize traditional MySQL Community Dinner event during Tutorial Day of the conference (April 10th)

SkySQL and MariaDB are hosting SkySQL Solutions Day (April 13) which will feature keynotes by Michael (Monty) Widenius and David Axmark.

Drizzle will have its 4th Annual Drizzle day (April 13)

Finally there is a first Sphinx Search Day hosted by Sphinx Technologies (April 13)

All these events are free to attend (registration needed) whenever you’re attending MySQL Conference and Expo or not.

Furthermore Birds of the Feather sessions on Percona Live MySQL Conference and Expo, taking place Tuesday and Wednesday in the evening are free for anyone to atten.

Categories: Libdrizzle News

Official Drizzle Blog: Drizzle 7.1 Release Candidate 7.1.32-rc has been released

Sat, 03/24/2012 - 11:38

This is the third Release candidate for Drizzle 7.1

Tar ball, Debian package and RPMs available.
Changes since previous release include:

* Various bug fixes
* Updated docs. Check out comprehensive replication documentation!

Changes in Drizzle 7.1 series since Drizzle 7 include:

- Multi-master replication
- Servers are identified with UUID in replication
- HTTP JSON interface available (experimental)
- Percona Innodb patches merged
- Xtrabackup in-tree
- IPV6 data type available
- Query log plugin
- Log output to syslog is enabled by default
- Ability to publish transactions to zeromq and rabbitmq
- Improvements to logging stats plugin
- Native multi-tenancy (database virtualization) support (catalogs)
- Removal of drizzleadmin utility (you can now do all administration from drizzle client itself)
- Removal of HailDB engine
- Revamped testing system Kewpie all-inclusive with suites of randgen, sysbench, sql-bench, and crashme tests
- Continued code refactoring & various bug fixes

Installing Drizzle 7.1.32-rc

The source tar package is available from our Launchpad download area. Please see the manual on how to install Drizzle 7.1.32-rc from source tar package.

The Ubuntu packages are available from the Drizzle PPA. Instructions on how to install from the PPA are in the Drizzle manual.

CentOS 6 packages are available from download.drizzle.org. Instructions on how to install them with yum are in the Drizzle manual.

Note: For Drizzle 7.1.32-rc, only 64-bit packages were built for RedHat/Centos. Please let us know if there is demand for 32-bit packages (such as by commenting to this blog post.)

Known issues

All of the optional plugins are packaged in their own package. These are named drizzle-plugin-*. If a plugin is installed, it is also configured to load by default. (see /etc/drizzle/conf.d/pluginname.cnf) However, not all plugins actually work without further configuration. These plugins will cause the server startup to fail. The workaround is to either configure the plugin to work, or uninstall it if you don't want to use it. (Related blueprint) In short, you should only install the plugins you want to use. Don't install all available plugins.

Please post bugs on the Drizzle bug tracker and questions and general feedback to drizzle-discuss mailing list.

Categories: Libdrizzle News

MySQL Performance Blog: Troubleshooting MySQL Memory Usage

Wed, 03/21/2012 - 13:09

One of the most painful troubleshooting tasks with MySQL is troubleshooting memory usage. The problem usually starts like this – you have configured MySQL to use reasonable global buffers, such as
innodb_buffer_size, key_buffer_size etc, you have reasonable amount of connections but yet MySQL takes much more memory than you would expect, causing swapping or other problems.

This simple problem on the surface becomes challenge with MySQL because there are no clear resource usage metrics available, and so in most cases you do not know where exactly memory is allocated. This was not much of the problem in MySQL 3.23 when there would only handful of places where memory could be allocated but it is a lot larger problems with MySQL 5.5 with addition of user variables, stored procedures, prepared statements etc which can be a memory hog.

My intent with this post is dual. I would encourage MySQL Team at Oracle, MariaDB team or Drizzle team to take a look into solving this problem. You will be thanked by a lot of people running MySQL in wild. I also wanted to share some troubleshooting techniques I use.

Plot Memory Usage

First I would like to see MySQL memory consumption plotted. I use “VSZ” columns from “ps” output on Linux. It helps me to understand how this memory allocation happens – does it grows slowly and never ending which would correspond to memory leak or resource leak, does it spikes at certain times which I may be able to correlate to some events ?

If you can’t get fancy graph quickly you can use this simple script:

while true do date >> ps.log ps aux | grep mysqld >> ps.log sleep 60 done

Check for Table Cache Related Allocations

There are cases when MySQL will allocate a lot of memory for table cache, especially if you’re using large blobs. It is easy to check though. Run “FLUSH TABLES” and see whenever memory usage goes down. Note though because of how memory is allocated from OS you might not see “VSZ” going down. What you might see instead is flushing tables regularly or reducing table cache reduces memory consumption to be withing the reason.

Connection Related Allocations
Another set of buffers correspond to connections – orphaned prepared statements, user variables, huge network buffer (can grow up to max_packet_size per connection) are all connection buffers and so if you close connection MySQL can clean them up. Killing connections (or stopping related applications) and observing whenever memory usage shrinks can be very helpful in diagnostics. If you have many applications it might make sense to shut them down one by one so it is easier to understand which of the applications were responsible for memory allocation.

If you figured out it is related to connections and identified application which causes excessive memory usage you might look at how it uses MySQL to identify potential causes. Is it working with large blobs ? Using user variables ? Prepared Statements ? memory tables ? In a lot of cases you have to guess and test, you can’t get information of how much memory connection uses and for which purposes it is allocated. Reducing various per connection variables is how you can test it, though it does not cover everything. For prepared statements you might want to look at Prepared_stmt_count to see how many prepared statements are allocated on server and Com_stmt_send_long_data to see whenever sending long data feature is used as it can increase amount of resources prepared statements take on server. There is no comparable variables of how many user variables are allocated (and how much memory they use).

Memory Tables

MEMORY tables can take memory. There are implicit MEMORY tables which are allocated for query execution, which size can be controlled by tmp_table_size and which also only exist for duration of query execution so it is usually easy to catch them. There are also explicit MEMORY tables you can create both as permanent and temporary. There is a max_heap_table_size variable which allows you to limit size of MEMORY tables (the limit applies both to implicit and explicit ones) but as there is no control of how many tables application can create it does not really allows to restrict memory usage. For Permanent tables it is easy. We can look at information_schema to see how much memory is being used by current MEMORY tables:

mysql> select sum(data_length+index_length) from information_schema.tables where engine='memory'; +-------------------------------+ | sum(data_length+index_length) | +-------------------------------+ | 126984 | +-------------------------------+ 1 row in set (0.98 sec)

This however will not cover TEMPORARY tables some of your connections might have created and forgot to remove (or still use for processing). Of course you will see these tables going away if you close connection. In Percona Server you can do better as you can query temporary tables too:

mysql> select sum(data_length+index_length) from information_schema.global_temporary_tables where engine='memory'; +-------------------------------+ | sum(data_length+index_length) | +-------------------------------+ | 126984 | +-------------------------------+ 1 row in set (0.00 sec)

You can even go deeper to check which sessions have created which temporary tables (both in memory and not):

mysql> select * from information_schema.global_temporary_tables \G *************************** 1. row *************************** SESSION_ID: 7234 TABLE_SCHEMA: test TABLE_NAME: my ENGINE: InnoDB NAME: #sql516_1c42_2 TABLE_ROWS: 0 AVG_ROW_LENGTH: 0 DATA_LENGTH: 16384 INDEX_LENGTH: 0 CREATE_TIME: NULL UPDATE_TIME: NULL *************************** 2. row *************************** SESSION_ID: 7234 TABLE_SCHEMA: test TABLE_NAME: tmp ENGINE: MEMORY NAME: #sql516_1c42_1 TABLE_ROWS: 2 AVG_ROW_LENGTH: 257 DATA_LENGTH: 126984 INDEX_LENGTH: 0 CREATE_TIME: 2012-03-21 05:52:02 UPDATE_TIME: NULL *************************** 3. row *************************** SESSION_ID: 7231 TABLE_SCHEMA: test TABLE_NAME: z ENGINE: InnoDB NAME: #sql516_1c3f_0 TABLE_ROWS: 0 AVG_ROW_LENGTH: 0 DATA_LENGTH: 16384 INDEX_LENGTH: 0 CREATE_TIME: NULL UPDATE_TIME: NULL 3 rows in set (0.00 sec)

Innodb Memory Usage

Finally it is often helpful to check how much memory Innodb has allocated. In fact this is often one of the first things I do as it is least intrusive. Run SHOW ENGINE INNODB STATUS and look for memory information block, which can use like this:

---------------------- BUFFER POOL AND MEMORY ---------------------- Total memory allocated 132183490560; in additional pool allocated 0 Internal hash tables (constant factor + variable factor) Adaptive hash index 4422068288 (2039977928 + 2382090360) Page hash 127499384 Dictionary cache 512619219 (509995888 + 2623331) File system 294352 (82672 + 211680) Lock system 318875832 (318747272 + 128560) Recovery system 0 (0 + 0) Threads 425080 (406936 + 18144) Dictionary memory allocated 2623331 Buffer pool size 7864319 Buffer pool size, bytes 128849002496 Free buffers 1 Database pages 8252672 Old database pages 3046376 Modified db pages 23419

I’m using the information from Percona Server again which provides a lot more insight about how memory is used. I would note though the output is a bit confusing as “Total Memory Allocated” is really not the total any more as Innodb has moved allocating memory from operation system directly not from addition memory pool (hence 0). Such allocations are not seen in “Total memory allocated” line yet they are seen in one of the lines below. The most likely cause for Innodb run away memory allocation is Dictionary cache though there could be other reasons too.

Memory Leaks
There are many kinds of memory leaks and I would say these are rather rare in MySQL. Most suspected memory leaks end up being some run away resource usage, though these can happen. Some memory leaks might happen per connection and they will be gone when connection is closed, other correspond to global memory allocation and will result in increased memory allocation until server is restarted. I would suspect memory leak when you see memory usage growing which can’t be connected to any known resource use. For example for global memory leaks you would see memory usage continues to grow even if you close connections and tables regularly. Another common thing about memory leaks is because it is memory which is allocated and forgotten about, unless it is very small blocks, it should be just swapped out and when never needed again. So if you see swap file used space gradually growing and there are “swap outs” but a lot less “swap ins” chances are it is caused by memory leak.

Dealing with memory leaks is rather complicated as good tools to detect memory leaks like valgrind are too slow to run in production. So the best thing to do in this case is see whenever you can create isolated repeatable test cases based on your application, which can illustrate memory leak and when it can be found and fixed. This is where your MySQL Support contract can be handy.

Conclusion
Understanding where MySQL can allocate memory can help us to find the cause in most cases. It is not as straightforward as it should be and I’m very hopeful future releases of MySQL, MariaDB or Drizzle bring improvements in this space allowing us to see directly for what purpose memory is allocated and so detect all kinds of memory usage problems easier.

Categories: Libdrizzle News

Henrik Ingo: IPv6 in Drizzle, soon in MySQL?

Sun, 03/18/2012 - 22:35

Earlier today I posted a Drizzle white paper we've been working on: Drizzle and IPv6.

read more

Categories: Libdrizzle News

Official Drizzle Blog: Drizzle White Paper published: Drizzle and IPv6

Sun, 03/18/2012 - 08:51

Drizzle white papers highlight unique features and designs in Drizzle, and give you examples of how to use them. We are proud to announce that we have today published our first white paper: Drizzle and IPv6.

This white paper will explain how Drizzle supports IPv6. The support for IPv6 networking was added to Drizzle early on and is available already in the first Drizzle 7 release. The Drizzle 7.1 release also adds the support for an IPV6 Data Type, to provide a way of storing IPv6 addresses in the database that is both user friendly and efficient. As an introduction we will also cover the differences between IPv4 and IPv6 network addresses and the reasons for transitioning from IPv4 to IPv6 addresses.

With this white paper we wish to thank the Google Summer of Code program for supporting Drizzle. The IPV6 data type was developed as a GSOC 2011 project by Muhammad Umair, with Mark Atwood mentoring. The IPv6 networking support was developed by Brian Aker and Eric Day, at the time employed by Sun Microsystems.

Categories: Libdrizzle News

Vijay Samuel: vjsamuel

Fri, 03/16/2012 - 18:53

Great news for all Google Summer of Code aspirants. Drizzle has been accepted as a mentoring organization for the 4th time. Students are welcome to look at our wiki page link and find possible projects and mentors. We hangout at #drizzle on IRC Freenode. Feel free to drop by and post your queries. We have devs from across the globe and so you may get your queries solved at the earliest. If you cant find us on the IRC channel then just shoot out a mail to our mailing list Drizzle-Discuss. We have good documentation on how to get started with code contribution. We would love to see you submit patches for low hanging fruit when you try to get a fix on your project proposals. We hope to have good talent at Drizzle this year. Put on those thinking hats and get those ready. Happy hacking fellow students!!!


Categories: Libdrizzle News

Daniel Nichter: How to find bugs in Drizzle slave replication

Thu, 03/15/2012 - 18:28
Number one on my second list of what Drizzle needs is replication documentation.  So I’ve been completely rewriting the replication documentation for Drizzle 7.1.  This is critical, in my humble opinion, not only because complete and accurate replication docu is needed, but also because I’m giving a talk “Getting Started with Drizzle 7.1” at the MySQL [...]
Categories: Libdrizzle News

Robert Hodges: Presenting at Percona Live and SkySQL MariaDB Solutions Day in Santa Clara

Mon, 03/12/2012 - 22:53
MySQL community conferences are alive and well in 2012.   Percona has taken the initiative to host the yearly MySQL event at the Santa Clara Hyatt; it's now called Percona Live MySQL Conference and Expo.  It runs from 10 through 12 April.  But don't plan on going home Thursday night.  On Friday 13 April you can also attend the SkySQL and MariaDB MySQL Solutions Day in the same location.  And wait, that's not all!  Drizzle Day is also on 13 April and also at the Hyatt, so you can catch up on what the Drizzle folks have been up to for the last 12 months.

Now for some specifics on the conferences where Continuent will be appearing.  Percona did an outstanding job of organizing conferences in 2011 and has a strong line-up for the 2012 Santa Clara event that includes a number of talks related to Tungsten.  Several of my Continuent colleagues, including Ed Archibald, Jeff Mace, and Giuseppe Maxia will be presenting.  I'm also pleased to report that the committee selected two of my proposals:

* Be a Data Management Hero with Good Backups! -- My worst mistake in 35 years of working in IT involved backups.  It's important to get them right.

Boost Your Replication Throughput with Parallel Apply, Prefetch, and Batching (with my colleague Stephane Giron, aka the master of the MySQL binlog)  -- We will cover how Tungsten speeds up replication in multiple ways, including new techniques like automatic batch loading that have not previously been available to MySQL users.

There will likely be a BOF for Tungsten--not confirmed but I am hopeful--on one of the evenings.    Please stop by if you would like to meet the engineers, and talk about Tungsten.  Afterwards we'll buy you a drink.  It comes out of the Continuent marketing budget, which proves that free software sometimes is linked to free beer.

Meanwhile, the SkySQL and MariaDB Solutions Day is new, at least for me.  Giuseppe Maxia and I will be doing a talk on building scalable apps with Tungsten replication and clustering.  It will stress how to build very large MySQL installations using Continuent's products.  I am looking forward to this conference, because we don't see as much on MariaDB here in the US as we do on the Oracle and Percona flavors of MySQL.  This is a great way for US-based folks to catch up. I also like the fact it will cover open source and commercial products like Tungsten Enterprise (our software) that help users take full advantage of MySQL.  Not everything needs to be open source to be cost-effective and useful.

Last year I suggested that we all lay down the fish for a while to work on conferences that build the MySQL community and sustain innovation.  Continuent will support any conference that meets those simple criteria.  I'm delighted that we are able to help sponsor both Percona Live and the SkySQL and MariaDB Solution Day to make Santa Clara worth visiting in April.  I look forward to meeting a lot of old friends and making new ones.  Please stop by and introduce yourself if you attend.
Categories: Libdrizzle News

Stewart Smith: Sessions at the Percona Live MySQL Conference that interest me

Thu, 03/08/2012 - 23:33

For the past many years, there’s been a conference in April, at the Santa Clara Convention Centre where the topic has been MySQL and the surrounding ecosystem. The first year I went, I gave a talk on the new features in MySQL Cluster 5.1 to a overflowing room of attendees. For me, it’s an event that’s mixed with speaking about something I’ve been working on and talking to other attendees about everything from how a particular part of the server works to where we can escape to for nearby good vegan food.

So, I thought I’d share some of the sessions that I’m really looking forward to. My selection is probably atypical, but may be interesting to others. I’m not going to list the keynotes, although they are often of a lot of value. I’m also going to attempt to avoid listing a few really awesome well known speakers simply because there are other really interesting sessions that also need exposure!

  • Starring Sakila: Building Data Warehouses and BI solutions using MySQL and Pentaho
    I need to base decisions off data, not simply a gut feeling (I’m not Stephen Colbert after all). I ran into a bunch of stumbling blocks when trying to work with Pentaho a couple of weeks ago, and I’m really hoping that this session shines some light on how to use it to better and more easily make arguments based on evidence to others in the company.
  • Testing MySQL Databases: The State Of The Art
    I’ve worked with Patrick for several years now, and he’s currently a valuable member of my team at Percona. For those who are interested in the state of the art of open source database testing, this is the session to be in.
  • Getting InnoDB Compression Ready for Facebook Scale
    This session is on at the same time as I’m speaking, so I probably won’t be able to attend (people keep coming to my sessions so I usually can’t sneak out). I’m really interested in how they’ve modified the compression code to help with their (large) workload.
  • Backing Up Facebook
    I hear that Facebook has a couple of database servers, a few dozen users and a few floppy disks full of data. This should be a fun story :)
  • Introducing XtraBackup Manager
    Being responsible for XtraBackup development at Percona, the XtraBackup topics really interest me. Lachlan has been working on a simple backup manager for XtraBackup to help create something that is a more complete backup solution than a tool which simply creates a backup.
  • Extending Xtrabackup – A Point-In-Time System
    Another good case of using XtraBackup as part of a comprehensive backup strategy. I have to be honest, I’m looking for ways in which we can improve XtraBackup to better fit the needs of people. It may be that there are a few small things we can do to make it easier for people do deploy and use.
  • Getting Started with Drizzle 7.1
    We’re about to do the 7.1 release of Drizzle! If you’re interested in having a SQL database that is designed to be used in large scale web applications and cloud environments, come along to this talk.
  • MySQL Idiosyncrasies That Bite
    I have to admit, I’m interested in Ronalds talk here to basically ensure we didn’t miss fixing anything in Drizzle. I do promise not to at any point yell out “Fixed in Drizzle” though.

Go here to register: http://www.percona.com/live/mysql-conference-2012/ (early bird pricing and discounted hotel rooms end March 12th, so you want to register sooner rather than later).

Categories: Libdrizzle News

Brian Aker: MySQL Conference, Percona, the Ecosystem...

Thu, 03/08/2012 - 07:38
This year O'Reilly isn't running the MySQL conference, Percona is doing it. Santa Clara, April 10-12. The usual time and place.

This is great news. They run a great conference. They always get rave reviews.

This isn't the first one they've done. They've done San Francisco, New York, London, more.

What's different about Santa Clara?

This isn't their typical 300-person one-day conference. They're picking up the annual MySQL users conference and carrying the torch forward. This is a 1000+ person, huge expo hall, 8 concurrent tracks, several days conference.

This year is a banner year for the MySQL ecosystem.  Why?

Because now the conference is focused on MySQL again. This is the central conference for the ecosystem. And now it's managed by a company that understands both the community and business side.

The sessions? Better than ever. In the past there was a lot of auxiliary content. Take a look. This year it's easily the best technical content I've ever seen. And I've led the selection committee for years.

The sponsors and exhibitors? Pretty much anyone who's anyone will be there.  Looking for solutions to your MySQL problems? You'll meet the folks who can help you. Visit the expo hall. There is great sponsor support for this year's event. HP, Facebook, Clustrix, Google, et cetera.

The keynotes are nothing to sneeze at too. There will be talks from Marten Mickos, Mark Callaghan, myself, and more of the old gang.

Is there more? Yes there is.

There are the community awards, BOFs, lightning talks. There is a Tuesday welcome reception, Wednesday community networking.

More? Yes. There is not one, not two, but three follow-on events. Stay an extra day and on Friday you can attend Drizzle DaySkySQL and MariaDB Day, or Sphinx Search Day.  Three awesome technologies for MySQL users.

This is a really big deal. If you are even slightly interested in MySQL, this is the event of the year. Don't miss it. By the way, early-bird pricing is almost expired. Register before March 12th or you'll pay more for the ticket and the hotel.
Categories: Libdrizzle News

Pages