Libzdb

Version 3.4.0

A small, easy to use Open Source Database Connection Pool Library with the following features:

  • Thread safe Database Connection Pool
  • Connect to multiple database systems simultaneously
  • Zero runtime configuration, connect using a URL scheme
  • Supports MySQL, PostgreSQL, SQLite and Oracle
Download

Requirements: Runs on iOS, Linux, macOS, FreeBSD, Solaris, OpenBSD and other POSIX systems. A C99 compiler is required to build the library.

Compatible with and can be included in C++ or Objective-C projects

Compatible with and can be included in C++ or Objective-C projects.

Modern, Object Oriented API design. Fully documented.

The library is licensed under a Free Open Source Software License.

Used in M/Monit

- The M/Monit Team mmonit.com

Used in DBMail

dbmail.org
Can I use libzdb in my iOS or macOS app?

Yes, libzdb can be used from and included in any C, C++ or Objective-C project. The Xcode project file used to develop libzdb is available from the repository and can be included in your own Xcode project.

Is the library thread-safe?

Libzdb is thread-safe and designed to be used in a multi-threaded program.

Is the connection pool dynamic?

Yes, the pool can be setup to dynamically change the number of Connections in the pool depending on the load.

Can I connect to multiple database systems at the same time?

Of course, libzdb is the perfect glue layer between different database systems or databases. For instance, you can read data from MySQL and write to PostgresSQL in just a few lines of code

Public code repository?

The project is hosted at Bitbucket. Click the r icon in the footer below to visit

Are there plans to support additional database systems?

Libzdb currently supports SQLite, MySQL, PostgreSQL and Oracle. At the moment there are no plans to support additional database systems.

C

To use libzdb in your C project (C11 or later), include zdb.h:

#include <zdb.h>
                            

Start by reading the documentation for the Connection Pool.

You can also view the file list directly. Each C module is extensively documented with example code.

Read below about the Connection URL that is used to specify the Database connection.

C++

To use libzdb in your C++ project (C++20 or later), import zdbpp.h and use the namespace zdb:

#include <zdbpp.h>
using namespace zdb;
                            

Start by reading the introduction to libzdb for C++. At the bottom of that page is an overview of the classes in the library.

You can also view the namespace class list directly. Each class is extensively documented with example code.

Connection URL:

The URL given to a Connection Pool at creation time specify a database connection on the standard URL format. The format of the connection URL is defined as:

database://[user:password@][host][:port]/database[?name=value][&name=value]...

The property names user and password are always recognized and specify how to login to the database. Other properties depends on the database server in question. User name and password can alternatively be specified in the auth-part of the URL. If port number is omitted, the default port number for the database server is used. Reserved characters used in the Connection URL must be URL encoded.

MySQL:

Here is an example on how to connect to a MySQL database server:

mysql://localhost:3306/test?user=root&password=swordfish

In this case the username, root and password, swordfish are specified as properties to the URL. An alternative is to use the auth-part of the URL to specify authentication information:

mysql://root:swordfish@localhost:3306/test

See mysql options for all properties that can be set for a mysql connection URL.

SQLite:

For a SQLite database the connection URL should simply specify a database file, since a SQLite database is just a file in the filesystem. SQLite uses pragma commands for performance tuning and other special purpose database commands. Pragma syntax on the form, name=value can be added as properties to the URL and will be set when the Connection is created. In addition to pragmas, the following properties are supported:

  • heap_limit=value [KB] - Make SQLite auto-release unused memory if memory usage goes above the specified value.
  • serialized=true - Make SQLite switch to serialized mode if value is true, otherwise multi-thread mode is used (the default).

An URL for connecting to a SQLite database might look like:

sqlite:///var/sqlite/test.db?synchronous=normal&foreign_keys=on&journal_mode=wal&temp_store=memory

PostgreSQL:

The URL for connecting to a PostgreSQL database server might look like:

postgresql://localhost:5432/test?user=root&password=swordfish

As with the MySQL URL, the username and password are specified as properties to the URL. Likewise, the auth-part of the URL can be used instead to specify the username and the password:

postgresql://root:swordfish@localhost/test?use-ssl=true

In this example we have also omitted the port number to the server, in which case the default port number, 5432, for PostgreSQL is used. In addition we have added an extra parameter to the URL, so connection to the server is done over a secure SSL connection.

See postgresql options for all properties that can be set for a postgresql connection URL.

Oracle:

The URL for connecting to an Oracle database server might look like:

oracle://localhost:1521/servicename?user=scott&password=tiger

Instead of a database name, Oracle uses a service name. The information in the url above is typically specified in a tnsnames.ora configuration file, pointed to by the environment variable TNS_ADMIN. In the example below, instead of host, port and service name, we use a tnsname as defined in tnsnames.ora. We also use the auth-part of the URL to specify the username and the password. Finally, we specify that we want to connect to Oracle with the SYSDBA role.

oracle://sys:secret@/tnsname?sysdba=true

See oracle options for all properties that can be set for an oracle connection URL.

Bridging Your Database Systems

Libzdb allows you to create multiple ConnectionPool objects, each connecting to different database systems as needed. This makes libzdb an excellent tool for querying, copying, and moving data between various database systems and instances.

The example below demonstrates this capability by copying data from a MySQL database to a PostgreSQL database. While simplified, it illustrates how libzdb can effectively serve as a connecting layer between your different databases.

Copying Apples to Oranges:

try {
    // Start a ConnectionPool to the Apples Warehouse Database 
    ConnectionPool apple_store("mysql://root:fruit@192.168.1.101:3306/apples");
    apple_store.start();

    // Start a ConnectionPool to the Oranges Warehouse Database                     
    ConnectionPool orange_store("postgresql://root:ninja@192.168.5.210/oranges");
    orange_store.start();

    // Get a Connection to the Orange store
    Connection orange_connection = orange_store.getConnection();

    // Select Apples we want to copy to Oranges
    Connection apple_connection = apple_store.getConnection();
    ResultSet apples = apple_connection.executeQuery(
        "SELECT name, color, weight FROM apples"
    );

    // Create a Prepared statement for storing Oranges
    PreparedStatement orange = orange_connection.prepareStatement(
        "INSERT INTO oranges (name, color, weight) VALUES (?, ?, ?)"
    );

    // Copy all Apples to Oranges under a transaction
    try {
        orange_connection.beginTransaction();
        while (apples.next()) {
            orange.bindValues(
                apples.getString("name").value_or(""),
                apples.getString("color").value_or(""),
                apples.getDouble("weight")
            );
            orange.execute();
        }
        orange_connection.commit();
        std::cout << "Transfer successful" << std::endl;
    } catch (const sql_exception& e) {
        orange_connection.rollback();
        std::cerr << "Transfer failed: " << e.what() << std::endl;
    }

    // Connections are automatically returned to their respective pools when 
    // they go out of scope
} catch (const sql_exception& e) {
    std::cerr << "Database error: " << e.what() << std::endl;
}
                    

Version 3.4.0

Released on 1 August 2024
  • New: Connection_beginTransactionType() can be used to specify a transaction's isolation level and characteristics when beginning a new transaction.
  • New: valueOr macro for safe handling of API return values:
    • Provides a default for NULL, 0, or negative returns.
    • Works with pointers, integers, and floats.
    • Safely evaluates expressions only once.
    • Compatible with GCC and Clang (uses GNU C extension).
    Example: const char* host = valueOr(URL_getHost(url), "localhost");
  • New: Enabled multi-thread mode for SQLite connections by default. This can improve performance in multi-threaded applications by allowing concurrent access to the database. Works with stock libsqlite builds (SQLITE_THREADSAFE=1 or 2). The URL property, serialized=true, can be used to set the connection back in serialized thread mode.
  • New: Added PreparedStatement_setSString() for setting a sized string value. If the string length is known, using this method is more efficient.
  • Fixed: Disabled use of sqlite3_enable_shared_cache and SQLITE_OPEN_SHAREDCACHE with open_v2(). Although we did see some concurrency benefits with shared cache in the past, it is best not to use it due to some potential nasty side-effects. SQLite also discourages its use. Luckily, stock libsqlite is built with shared cache disabled so our attempt to activate it was essentially a no-op in any case.
  • Fixed: C++, added missing support for binding a floating point value in PreparedStatement::bind()
  • Fixed: C++, Use PreparedStatement_setSString() when binding a std::string_view which is more efficient and safer.
  • Fixed: Plus minor improvements and fixes

Version 3.3.0

Released on 23 July 2024
  • New: The pool now starts the reaper thread by default, while previously it was started on request. The function ConnectionPool_setReaper() has changed purpose from previously starting the reaper thread to now being able to disable the reaper thread by setting sweep interval to 0.
  • New: The C++ API has been updated to have better ergonomics and provide more modern features. C++20 is required, and the API is not backward compatible. See test/zdbpp.cpp for examples and use of libzdb in a C++ project. The C API remains backward compatible and does not require any changes to your code.
  • New: Added ConnectionPool_isFull() for improved connection pool management. This method allows users to check if the pool is at capacity before attempting to get a connection. It's recommended to use ConnectionPool_isFull() before calling ConnectionPool_getConnection() to proactively manage pool resources and handle full pool scenarios efficiently, such as increasing max connections
  • New: Added PreparedStatement_setNull() to explicit set a parameter in a prepared statements to SQL NULL.
  • New: Changed default timeout before an inactive connection is evicted from the pool, from 30 seconds to 90 seconds. ConnectionPool_setConnectionTimeout() can be used to change this value.
  • Fixed: Improved Connection Pool Concurrency and Reliability. Significantly reduced lock contention in the connection pool when multiple threads request connections simultaneously. Connections returned from the pool are still guaranteed to be active and connected to the database.
  • Fixed: Plus minor improvements and fixes

Version 3.2.3

Released on 18 October 2022
  • New: Provide better error reporting if a Connection cannot be obtained from the Connection Pool.
  • Fixed: In C++, added a guard against stopping the Connection Pool with live active Connections to prevent a dealloc sequence problem.
  • Fixed: Plus many more minor improvements and fixes

Version 3.2.2

Released on 1 April 2020
  • Fixed: Removed Thread.h from the public API.

Version 3.2.1

Released on 6 March 2020
  • New: Include Library version number in zdb.h
  • Fixed: Simplified test/zdbpp.cpp and added missing header
  • Fixed: Improved support for MySQL 8 and MariaDB

Version 3.2

Released on 3 Apr 2019
  • New: C++17 support via zdbpp.h which is distributed with libzdb for more idiomatic use of libzdb from C++. Thanks to dragon jiang (jianlinlong)
  • New: Support prefetch rows for MySQL and Oracle. Either programatically via Connection_setFetchSize() or via ResultSet_setFetchSize() or via a new global fetch-size URL option. Thanks to dragon jiang (jianlinlong)
  • New: MySQL 5.7 and later. Added session query timeout accessible via Connection_setQueryTimeout()
  • New: MySQL 8. Added a new URL option auth-plugin which specify the authentication plugin to use when connecting to a MySQL server.
  • New: Oracle: Added a new URL option sysdba for connecting with sysdba privileges.
  • Fixed: Revert previous fix (#8) and remove last SQL terminator character ';' in statements, except if preceded with END; to allow for ending a pl/sql block.
  • Fixed: Oracle: Set SQL null value in prepared statement
  • Fixed: Oracle: Handle date/time literal values

Version 3.1

Released on 31 Aug 2015
  • New: Support Literal IPv6 Addresses in the Database Connection URL. Ref. RFC2732
  • New: Honour timezone information if provided with date-time column values in Result Sets
  • Fixed: Issue #7 Removed onstop handler
  • Fixed: #8 Do not remove trailing SQL termination charachter ';' from statement

Version 3.0

Released on 06 Jan 2014
  • New: Methods for retrieving Date, Time, DateTime and TimeStamp column values from a ResultSet. PreparedStatement_setTimestamp for setting Unix timestamp.
  • New: ResultSet_isnull, can be used to explicit test if a column value in a Result Set is SQL null. A Result Set already returns the NULL pointer for string and blob types and 0 for primitive data types if column value is SQL null, but to differ between SQL null and the value NULL or 0, this method can be used.
  • New: PreparedStatement_getParameterCount, Returns the number of parameters in a prepared statement
  • New: It is now a checked runtime error for the url parameter given in ConnectionPool_new to be NULL.
  • New: No longer require flex installed as the generated file is part of the distribution.
  • Fixed: Oracle: memory corruption in OracleResultSet when a Blob value is retrieved as a String

Version 2.12

Released on 03 Sep 2013
  • New: PreparedStatement_rowsChanged added to PreparedStatement.h
  • Fixed: Oracle: OCIPing is used to check Oracle connections and to ensure that the Pool returns connected connections. Thanks to Pavlo Lavrenenko.

Version 2.11/3

Released on 05 Jun 2013
  • New: License Exception added to allow the library to be linked and distributed together with OpenSSL.
  • New: Throw SQLException if a database access error occurs when ResultSet_next() is called. Previously, access errors could be masked as end of result set. Thanks to JiaQiang Xu.
  • Fixed: (Volodymyr Tarasenko) Possible mem leak in Oracle's blob operation fixed.
  • Fixed: MySQL: A ResultSet bind memory error could occur if string or blob columns of different size caused successive buffer reallocation. Thanks to Ryan Addams for discovering the problem.
  • New: Added support for the new bytea hex encoding format introduced in PostgreSQL 9.0.
  • Fixed: MySQL: A Result Set with two or more columns larger than 256 bytes would cause libzdb to truncate the second column to 256 bytes in the first row. Thanks to Me from China for bug report and patch.
  • Fixed: Improved Build Configuration. Thanks to Johan Bergström for report.

Version 2.10/.6

Released on 29 Oct 2012
  • New: Libzdb is now compatible with and can be included in C++ or Objective-C(++) projects.
  • Fixed: Oracle: Fixed a connection parameter bug. Thanks to l00106600
  • Fixed: Oracle: Fixed a GCC C99 compile issue. Thanks to Stas Oginsky
  • New: MySQL: Improved error reporting
  • New: Automatically unescape the following URL components: credentials, path and parameter values
  • New: Connection Pool start now throws an SQLException instead of calling abort handler if starting the pool failed. Thanks to Christopher O'Hara
  • Fixed: MySQL: Using a stored procedure which returned a result set would freeze the connection until it was reaped. Thanks to Jesse White
  • Fixed: MySQL: Ensure that the library can be restarted after it was stopped without leaking resources. Only applicable for MySQL which could leak memory on runtime restart. Thanks to Axel Steiner

Version 2.9

Released on 15 Aug 2011
  • New: SQLite: Unescape path to allow for (white-)space in database file URL path. Thanks to Jonas Schnelli
  • Fixed: SQLite: Use sqlite3_open_v2 instead of sqlite3_enable_shared_cache which is deprecated in OS X Lion
  • Fixed: Oracle: Fixed a problem with ResultSet not returning all data.

Version 2.8/.1

Released on 15 Feb 2011
  • New: PostgreSQL: Allow sending application name to the server for logging. Thanks to Chris Mayo. See the PostgreSQL URL property, application-name.
  • Fixed: Oracle: Fixed a ResultSet memory leak
  • Fixed: Oracle: Fixed a transaction related memory leak

Questions or comments?

If you have questions or comments about the software or documentation please subscribe to the libzdb general mailing list and post your questions there.

Open Source

Libzdb is open source. It's hosted, developed, and maintained on Bitbucket.

Reporting a bug

If you believe you have found a bug, please use the issue tracker to report the problem. Remember to include the necessary information that will enable us to understand and reproduce this problem.