Archive for andrew

MySQL Cluster 7.0.8 source released

7.0.8aa source release is now available from ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/

The source version for MySQL Cluster 7.0.8 has now been made available at ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/mysql-5.1.37-ndb-7.0.8/

You can either wait for the binaries to be released or if you’re in a rush then you can find instructions on building the binaries for yourself in the earlier article: “MySQL Cluster 7.0.7 source released“.

A description of all of the changes (fixes) that have gone into MySQL Cluster 7.0.8 (compared to 7.0.7) can be found in the MySQL Cluster 7.0.8 Change Log.





Another free webinar today – FreeRADIUS & MySQL Cluster: Scalable and Highly Available AAA Services

As network use grows and services become more dynamic, so existing Authentication, Authorization and Accounting (AAA) environments can struggle to keep pace with demand.

Tune into this webinar where you can hear from the Alan Dekok, one of the founders of the FreeRADIUS project and CEO of Network RADIUS, discuss the concepts and implementation of RADIUS services using the FreeRADIUS server and the MySQL Cluster database to deliver highly available and scalable AAA services.

As always, this webinar is free and you can register here. I will be manning the Q&A during the webinar.

In this session, you will learn about:

  • potential AAA limitations as network environments grow
  • advantages of deploying FreeRADIUS with MySQL Cluster
  • Performance, sizing and deployment of an AAA environment using FreeRADIUS with MySQL Cluster
  • customer case studies
  • how to get started

    WHO:

    Alan Dekok, Founder of the FreeRADIUS project and CEO of Network RADIUS
    Matthew Keep, MySQL Cluster Product Management

    WHAT:

    FreeRADIUS & MySQL Cluster: Scalable and Highly Available AAA Services web presentation.

    WHEN:

    Wednesday, September 30, 2009: 09:30 Pacific time (America)
    Wed, Sep 30: 06:30 Hawaii time
    Wed, Sep 30: 10:30 Mountain time (America)
    Wed, Sep 30: 11:30 Central time (America)
    Wed, Sep 30: 12:30 Eastern time (America)
    Wed, Sep 30: 16:30 UTC
    Wed, Sep 30: 17:30 Western European time
    Wed, Sep 30: 18:30 Central European time
    Wed, Sep 30: 19:30 Eastern European time

    The presentation will be approximately 45 minutes long followed by Q&A.

    WHERE:

    Simply access the web seminar from the comfort of your own office.

    WHY:

    To learn more about how you can use FreeRADIUS and MySQL Cluster 7.0





    Free MySQL webinar today – High Availability Architectures for Online Applications

    Update: You can now download a recording of the webinar and the slides from http://www.mysql.com/news-and-events/on-demand-webinars/display-od-403.html

    I’ll be presenting the fourth (and final) session of the MySQL for Online Applications webinar series today (29 September). Today’s High Availability Architectures for Online Applications webinar covers:

    • MySQL Replication
    • MySQL Cluster
    • Distributed Replicated Block Device (DRBD)
    • Other high-availability technologies

    Register here.

    This session starts at 10:00 am Pacific Time but will be rerun tomorrow at 10:00 am CET (9:00 am UK) with Ivan Zoratti presenting (I’ll be handling questions).

    More details for today’s webinar:

    Tuesday, September 29, 2009

    Join us for the last of our 4 part webinar series exploring the different aspects of using MySQL as the backend database for online applications. With real life experience gained working with MySQL Customers such as Facebook, Alcatel Lucent and Google, this webinar series will give you the information you need to run scalable, highly available online applications.

    In this last installment we look at MySQL high availability technologies and architectures. We will explore the uses cases for implementing:

    • MySQL Replication
    • MySQL Cluster
    • Distributed Replicated Block Device (DRBD)
    • Other high-availability technologies

    Also, covered will be the fundamentals of how these technologies work and how they can be combined to create a more scalable and highly available database infrastructure. Several case studies will be presented to show how these technologies have been implemented in the real world.

    Whether you are already using MySQL for your online application or considering it for a new project then register today to learn how you can make best use of the world’s most popular database for online applications.

    WHO:

    Andrew Morgan, Senior Product Manager, MySQL

    WHAT:

    High Availability Architectures for Online Applications web presentation.

    WHEN:

    Tuesday, September 29, 2009: 10:00 Pacific time (America)

    Tue, Sep 29: 07:00 Hawaii time
    Tue, Sep 29: 11:00 Mountain time (America)
    Tue, Sep 29: 12:00 Central time (America)
    Tue, Sep 29: 13:00 Eastern time (America)
    Tue, Sep 29: 17:00 UTC
    Tue, Sep 29: 18:00 Western European time
    Tue, Sep 29: 19:00 Central European time
    Tue, Sep 29: 20:00 Eastern European time
    High Availability Architectures for Online Applications




    MySQL Cluster Helps Mapion to Improve its User Experience

    A new case study has been published that describes how and why Mapion adopted MySQL Cluster as the real-time, mission critical database for their Mapion Mobile applications.

    Mapion adopted MySQL Cluster Databse because it allows for the availability of existing applications through its ‘shared-nothing’ distributed architecture with no single point of failure, to meet Mapion’s mission-critical application requirements of 99.999% availability. This is coupled with automatic data partitioning with load balancing, which allows almost unlimited database scalability for the company. Mr. Takashi Ando comments, “Some databases supported clusters when we previously reviewed our database technology, but they were hard to operate and would have increased costs substantially. We found ourselves in a completely different situation when planning the introduction of the new system. MySQL Cluster Database had made it possible to construct a highly available system while keeping costs down.”

    Mapion’s online map service enables users to search for addresses as well as gain access to navigation information such as directions for driving and public transport routes. The fee-based service for mobile users, Mapion Mobile, sends maps and directions direct to users’ mobile phones, while the Mapion BB broadband service offers more detailed maps, which smoothly and instantaneously refresh as the user navigates around the displayed map.

    Headquartered in Tokyo, Japan, Mapion Co. Ltd delivers Japan’s leading online map search service, with over 700,000 page views per day, and over 200 enterprise customers. Mapion’s shareholders include Toppan, NTT East, Dentsu, Yahoo Japan, and Sharp.
    The company’s online map service enables users to search for addresses as well as gain access to navigation information such as directions for driving and public transport routes. The fee-based service for mobile users, Mapion Mobile, sends maps and directions direct to users’ mobile phones, while the Mapion BB broadband service offers more detailed maps, which smoothly and instantaneously refresh as the user navigates around the displayed map




    Free Guide released: Scalable Authentication Services with FreeRADIUS and MySQL Cluster

    MySQL/Sun have releases a new white paper…

    Do you want to ensure that your Authentication, Authorization and Accounting (AAA) infrastructure will scale to support your business growth?
    As network use grows and services become more dynamic, limitations can occur which add administrative overhead, inhibit flexible scaling and impact the timely synchronization of data across the AAA environment.
    To address these challenges, Sun has collaborated with the FreeRADIUS server team, the most widely deployed RADIUS server in the world, to integrate the carrier-grade, real-time MySQL Cluster database with the FreeRADIUS Server.
    Delivering Scalable Authentication Services:
    Get the whitepaper now!
    Attend Webinar
    Download our free whitepaper “Delivering Scalable and Highly Available Authentication, Authorization and Accounting Services” now to better understand:
    The concepts of current data storage solutions for AAA environments and their potential limitations as network use grows
    How you can implement an infrastructure for high growth and high availability with low complexity by deploying the FreeRADIUS server and MySQL Cluster
    How the solution performs in real world AAA environments via a user case study.

    Do you want to ensure that your Authentication, Authorization and Accounting (AAA) infrastructure will scale to support your business growth?

    As network use grows and services become more dynamic, limitations can occur which add administrative overhead, inhibit flexible scaling and impact the timely synchronization of data across the AAA environment.

    To address these challenges, Sun has collaborated with the FreeRADIUS server team, the most widely deployed RADIUS server in the world, to integrate the carrier-grade, real-time MySQL Cluster database with the FreeRADIUS Server.

    Download our free whitepaper “Delivering Scalable and Highly Available Authentication, Authorization and Accounting Services” now to better understand:

    • The concepts of current data storage solutions for AAA environments and their potential limitations as network use grows
    • How you can implement an infrastructure for high growth and high availability with low complexity by deploying the FreeRADIUS server and MySQL Cluster
    • How the solution performs in real world AAA environments via a user case study.




    MySQL Cluster 6.3.26 binaries released

    The binaries for MySQL Cluster 6.3.26 have now been released and can be downloaded from http://dev.mysql.com/downloads/cluster/6.3.html

    A description of all of the changes (fixes) that have gone into MySQL Cluster 6.3.26 (compared to 6.3.25) can be found in the MySQL_Cluster_6_3_26_ChangeLog.





    MySQL Cluster 7.0.7 binaries released

    The binaries for MySQL Cluster 7.0.7 have now been released and can be downloaded from http://dev.mysql.com/downloads/cluster/7.0.html

    A description of all of the changes (fixes) that have gone into MySQL Cluster 7.0.7 (compared to 7.0.6) can be found in the MySQL Cluster 7.0.7 Change Log.





    MySQL Cluster 7.0.7 source released

    Update: As explained in “MySQL Cluster 7.0.7 binaries released” you can now download the compiled binaries for your particular platform. I am going to leave this entry in place as it will hopefully be useful for future releases but for 7.0.7 you should refer to that other post.

    You’ll need to wait for the pre-built binaries but you can now download the source code and build it for yourself to get started. This article explains where to get it and how to build, install and test the installation.

    In this example, I’ve used Ubuntu.

    The first step is to download the compressed tar ball containing the source code by pointing your browser to ftp://ftp.mysql.com/pub/mysql/download/cluster_telco/mysql-5.1.35-ndb-7.0.7/mysql-cluster-gpl-7.0.7.tar.gz

    Then you need to extract the code and then build and install the software:

    billy@billy-laptop:~/mysql$ tar -xzf mysql-cluster-gpl-7.0.7.tar.gz
    billy@billy-laptop:~/mysql$ mv mysql-cluster-gpl-7.0.7 7_0_7
    billy@billy-laptop:~/mysql$ cd 7_0_7
    billy@billy-laptop:~/mysql/7_0_7$ autoreconf --force --install
    billy@billy-laptop:~/mysql/7_0_7$ ./configure --with-plugins=max --prefix=/usr/local/mysql
    billy@billy-laptop:~/mysql/7_0_7$ make
    billy@billy-laptop:~/mysql/7_0_7$ sudo make install
    billy@billy-laptop:~/mysql/7_0_7$ sudo cp storage/ndb/src/kernel/ndbd /usr/local/mysql/bin
    billy@billy-laptop:~/mysql/7_0_7$ sudo cp storage/ndb/src/kernel/ndbmtd /usr/local/mysql/bin
    billy@billy-laptop:~/mysql/7_0_7$ sudo cp storage/ndb/src/mgmsrv/ndb_mgmd /usr/local/mysql/bin
    billy@billy-laptop:~/mysql/7_0_7$ cd /usr/local/mysql/mysql-test
    billy@billy-laptop:/usr/local/mysql/mysql-test$ sudo chmod 777 .
    billy@billy-laptop:/usr/local/mysql/mysql-test$ sudo chmod 777 *

    You can then test that the installation has been successful:

    billy@billy-laptop:/usr/local/mysql/mysql-test$ ./mtr ndb_basic
    
    Logging: ./mtr  ndb_basic
    090902 18:17:43 [Warning] Forcing shutdown of 2 plugins
    MySQL Version 5.1.35
    Checking supported features...
    - using ndbcluster when necessary, mysqld supports it
    - skipping SSL, mysqld not compiled with SSL
    - multi threaded ndbd, will be used "round robin"
    Collecting tests...
    vardir: /usr/local/mysql/mysql-test/var
    Removing old var directory...
    Creating var directory '/usr/local/mysql/mysql-test/var'...
    Installing system database...
    Using server port 43993
    
    ==============================================================================
    
    TEST                                      RESULT   TIME (ms)
    ------------------------------------------------------------
    
    worker[1] Using MTR_BUILD_THREAD 250, with reserved ports 12500..12509
    ndb.ndb_basic                            [ pass ]  73661
    ------------------------------------------------------------
    The servers were restarted 0 times
    Spent 73.661 of 143 seconds executing testcases
    
    All 1 tests were successful.

    Following this you can configure and use the software as normal (refer to Creating a simple Cluster on a single LINUX host).

    The annotated header information has already been updated in the NDB API Docs section.

    A description of all of the changes (fixes) that have gone into MySQL Cluster 7.0.7 (compared to 7.0.6) can be found in the MySQL Cluster 7.0.7 Change Log.





    MySQL Cluster: Geographic Replication Deep-Dive

    Following requests received during earlier MySQL Cluster webinars, a new (and as always, free) webinar has been scheduled which focuses on MySQL Cluster Replication. The webinar is scheduled for Thursday 10 September and you can register at http://www.mysql.com/news-and-events/web-seminars/display-415.html

    I’ll be on-line during the webinar, answering questions.

    Details….

    MySQL Cluster: Geographic Replication Deep-Dive

    Thursday, September 10, 2009

    MySQL Cluster has been deployed into some of the most demanding web, telecoms and enterprise / government workloads, supporting 99.999% availability with real time performance and linear write scalability.

    Tune into this webinar where you can hear from the Director of MySQL Server Engineering provide a detailed “deep dive” into one of MySQL Cluster’s key capabilities – Geographic Replication.

    In this session, you will learn how using Geographic Replication enables your applications to :

    • achieve higher levels of availability within a data center or across a WAN
    • locate data closer to users, providing lower latency access
    • replicate to other MySQL storage engines for complex data analysis and reporting of real time data
    • how to get started with Geographic Replication

    WHO:

    • Tomas Ulin, Director, MySQL Server Technologies
    • Matthew Keep, MySQL Cluster Product Management

    WHAT:

    MySQL Cluster: Geographic Replication Deep-Dive web presentation.

    WHEN:

    Thursday, September 10, 2009: 09:30 Pacific time (America)

    Thu, Sep 10: 06:30 Hawaii time
    Thu, Sep 10: 10:30 Mountain time (America)
    Thu, Sep 10: 11:30 Central time (America)
    Thu, Sep 10: 12:30 Eastern time (America)
    Thu, Sep 10: 16:30 UTC
    Thu, Sep 10: 17:30 Western European time
    Thu, Sep 10: 18:30 Central European time
    Thu, Sep 10: 19:30 Eastern European time

    The presentation will be approximately 45 minutes long followed by Q&A.

    WHERE:

    Simply access the web seminar from the comfort of your own office.

    WHY:

    To learn more about how you can use Geographic Replication in MySQL Cluster 7.0 to build real time, high performance applications delivering continuously available database services.





    Using NDB API Events to mask/hide colum data when replicating

    If you  have asynchronous replication where the slave database is using MySQL Cluster then you can use the NDB API events functionality to mask/overwrite data. You might do this for example if the replica is to be used for generating reports where some of the data is sensitive and not relevant to those reports. Unlike stored procedures, NDB API events will be triggered on the slave.

    The first step is to set up replication (master->slave rather than multi-master) as described in Setting up MySQL Asynchronous Replication for High Availability).

    In this example, the following table definition is used:

    mysql> use clusterdb;
    mysql> create table ASSETS (CODE int not null primary key, VALUE int) engine=ndb;

    The following code should be compiled and then executed on a node within the slave Cluster:

    #include <NdbApi.hpp>
    #include <stdio.h>
    #include <iostream>
    #include <unistd.h>
    #include <cstdlib>
    #include <string.h>
    
    #define APIERROR(error) 
      { std::cout << "Error in " << __FILE__ << ", line:" << __LINE__ << ", code:" 
      << error.code << ", msg: " << error.message << "." << std::endl; 
      exit(-1); }
    
    int myCreateEvent(Ndb* myNdb,
    const char *eventName,
    const char *eventTableName,
    const char **eventColumnName,
    const int noEventColumnName);
    
    static void do_blank(Ndb*, int);
    
    int main(int argc, char** argv)
    {
      if (argc < 1)
     {
        std::cout << "Arguments are <connect_string cluster>.n";
        exit(-1);
      }
      const char *connectstring = argv[1];
    
      ndb_init();
    
      Ndb_cluster_connection *cluster_connection=
      new Ndb_cluster_connection(connectstring); // Object representing the cluster
    
      int r= cluster_connection->connect(5 /* retries               */,
      3 /* delay between retries */,
      1 /* verbose               */);
      if (r > 0)
      {
        std::cout << "Cluster connect failed, possibly resolved with more retries.n";
        exit(-1);
      }
      else if (r < 0)
      {
        std::cout << "Cluster connect failed.n";
        exit(-1);
      }
    
      if (cluster_connection->wait_until_ready(30,30))
      {
        std::cout << "Cluster was not ready within 30 secs." << std::endl;
        exit(-1);
      }
    
      Ndb* myNdb= new Ndb(cluster_connection,
                          "clusterdb");  // Object representing the database
    
      if (myNdb->init() == -1) APIERROR(myNdb->getNdbError());
    
      const char *eventName= "CHNG_IN_ASSETS";
      const char *eventTableName= "ASSETS";
      const int noEventColumnName= 2;
      const char *eventColumnName[noEventColumnName]=
      {"CODE",
       "VALUE"};
    
      // Create events
      myCreateEvent(myNdb,
      eventName,
      eventTableName,
      eventColumnName,
      noEventColumnName);
    
      // Normal values and blobs are unfortunately handled differently..
      typedef union { NdbRecAttr* ra; NdbBlob* bh; } RA_BH;
    
      int i;
    
      // Start "transaction" for handling events
      NdbEventOperation* op;
      printf("create EventOperationn");
      if ((op = myNdb->createEventOperation(eventName)) == NULL)
        APIERROR(myNdb->getNdbError());
    
      printf("get valuesn");
      RA_BH recAttr[noEventColumnName];
      RA_BH recAttrPre[noEventColumnName];
    
      for (i = 0; i < noEventColumnName; i++) {
        recAttr[i].ra    = op->getValue(eventColumnName[i]);
        recAttrPre[i].ra = op->getPreValue(eventColumnName[i]);
      }
    
      // set up the callbacks
      // This starts changes to "start flowing"
      if (op->execute())
        APIERROR(op->getNdbError());
    
      while (true) {
        int r = myNdb->pollEvents(1000); // wait for event or 1000 ms
        if (r > 0) {
          while ((op= myNdb->nextEvent())) {
            NdbRecAttr* ra = recAttr[0].ra;
            if (ra->isNULL() >= 0) { // we have a value
              if (ra->isNULL() == 0) { // we have a non-null value
                printf("CODE: %d ", ra->u_32_value());
                do_blank(myNdb, ra->u_32_value());
              } else 
                printf("%-5s", "NULL");
              } else
                printf("%-5s", "-"); // no value
                ra = recAttr[1].ra;
                printf("n");
              }
            }
          }
        }
    
    int myCreateEvent(Ndb* myNdb,
                      const char *eventName,
                      const char *eventTableName,
                      const char **eventColumnNames,
                      const int noEventColumnNames)
    {
      NdbDictionary::Dictionary *myDict= myNdb->getDictionary();
      if (!myDict) APIERROR(myNdb->getNdbError());
    
      const NdbDictionary::Table *table= myDict->getTable(eventTableName);
      if (!table) APIERROR(myDict->getNdbError());
    
      NdbDictionary::Event myEvent(eventName, *table);
      myEvent.addTableEvent(NdbDictionary::Event::TE_INSERT);
    
      myEvent.addEventColumns(noEventColumnNames, eventColumnNames);
    
      // Add event to database
      if (myDict->createEvent(myEvent) == 0)
        myEvent.print();
      else if (myDict->getNdbError().classification ==
                NdbError::SchemaObjectExists) {
        printf("Event creation failed, event existsn");
        printf("dropping Event...n");
        if (myDict->dropEvent(eventName)) APIERROR(myDict->getNdbError());
        // try again
        // Add event to database
        if ( myDict->createEvent(myEvent)) APIERROR(myDict->getNdbError());
      } else
        APIERROR(myDict->getNdbError());
    
        return 0;
    }
    
    static void do_blank(Ndb* myNdb, int code)
    {
      const NdbDictionary::Dictionary* myDict= myNdb->getDictionary();
      const NdbDictionary::Table *myTable= myDict->getTable("ASSETS");
    
      if (myTable == NULL)
      APIERROR(myDict->getNdbError());
    
      NdbTransaction *myTransaction= myNdb->startTransaction();
      if (myTransaction == NULL) APIERROR(myNdb->getNdbError());
    
      printf("Replacing VALUE with 0 for CODE: %d ", code);
    
      NdbOperation *myOperation= myTransaction->getNdbOperation(myTable);
      if (myOperation == NULL) APIERROR(myTransaction->getNdbError());
    
      myOperation->updateTuple();
      myOperation->equal("CODE", code);
      myOperation->setValue("VALUE", 0);
    
      if (myTransaction->execute( NdbTransaction::Commit ) == -1)
        APIERROR(myTransaction->getNdbError());
    
      myNdb->closeTransaction(myTransaction);
    }
    
    shell> slave_filter 127.0.0.1:1186

    From the master Cluster, insert some values (note that the example can easily be extended to cover updates too):

    mysql> insert into ASSETS values (101, 50),(102, 40), (103, 99);

    and then check that on the slave the value has been set to 0 for each of the entries:

    mysql> select * from ASSETS;
    +------+-------+
    | CODE | VALUE |
    +------+-------+
    |  100 |     0 |
    |  103 |     0 |
    |  101 |     0 |
    |  102 |     0 |
    +------+-------+

    How this works…. The table data is replicated as normal and the real values are stored in the slave. The “slave_filter” process has registered against insert operations on this table and when it’s triggered it sets the VALUE field to 0. The event is processes asynchronously from the replication and so there will be some very narrow window during which the true values would be stored in the slave.