Edit File - File Manager

Edit File: txnexample_dpl.html

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>DPL Transaction Example</title>
    <link rel="stylesheet" href="gettingStarted.css" type="text/css" />
    <meta name="generator" content="DocBook XSL Stylesheets V1.62.4" />
    <link rel="home" href="index.html" title="Getting Started with Berkeley DB Transaction Processing" />
    <link rel="up" href="wrapup.html" title="Chapter 6. Summary and Examples" />
    <link rel="previous" href="txnexample_java.html" title="Base API Transaction Example" />
    <link rel="next" href="inmem_txnexample_java.html" title="Base API In-Memory Transaction Example" />
  </head>
  <body>
    <div class="navheader">
      <table width="100%" summary="Navigation header">
        <tr>
          <th colspan="3" align="center">DPL Transaction Example</th>
        </tr>
        <tr>
          <td width="20%" align="left"><a accesskey="p" href="txnexample_java.html">Prev</a> </td>
          <th width="60%" align="center">Chapter 6. Summary and Examples</th>
          <td width="20%" align="right"> <a accesskey="n" href="inmem_txnexample_java.html">Next</a></td>
        </tr>
      </table>
      <hr />
    </div>
    <div class="sect1" lang="en" xml:lang="en">
      <div class="titlepage">
        <div>
          <div>
            <h2 class="title" style="clear: both"><a id="txnexample_dpl"></a>DPL Transaction Example</h2>
          </div>
        </div>
        <div></div>
      </div>
      <p>
        The following Java code provides a fully functional example of a
        multi-threaded transactional DB application using the DPL. 
        This example is nearly identical to the example provided in the
        previous section, except that it uses an entity class and entity
        store to manage its data.
    </p>
      <p>
            As is the case with the previous examples, this example opens
            an environment and then an entity store. It then creates 
            5 threads, each of which writes 500 records to the database.
            The primary key for these writes are based on pre-determined
            integers, while the data is randomly generated data. 
            This means that the actual data is arbitrary and therefore uninteresting; 
            we picked it only because it requires minimum code to implement and therefore will 
        stay out of the way of the main points of this example.
    </p>
      <p>
        Each thread writes 10 records under a single transaction
        before committing and writing another 10 (this is repeated 50
        times). At the end of each transaction, but before committing, each
        thread calls a function that uses a cursor to read every record in
        the database. We do this in order to make some points about
        database reads in a transactional environment.
    </p>
      <p>
        Of course, each writer thread performs deadlock detection as
        described in this manual. In addition, normal recovery is performed
        when the environment is opened.
    </p>
      <p>
        To implement this example, we need three classes:    
    </p>
      <div class="itemizedlist">
        <ul type="disc">
          <li>
            <p>
                <tt class="literal">TxnGuide.java</tt>
            </p>
            <p>
                This is the main class for the application. It performs
                environment and store management, spawns threads, and
                creates the data that is placed in the database.  See <a href="txnexample_dpl.html#txnguideexample_dpl">TxnGuide.java</a> for implementation details.
            </p>
          </li>
          <li>
            <p>
                <tt class="literal">StoreWriter.java</tt>
            </p>
            <p>
                This class extends <tt class="literal">java.lang.Thread</tt>, and
                as such it is our thread implementation.  It is responsible
                for actually reading and writing store. It also
                performs all of our transaction management.  See <a href="txnexample_dpl.html#storewriter">StoreWriter.java</a> for
                implementation details.
            </p>
          </li>
          <li>
            <p>
                <tt class="literal">PayloadDataEntity.java</tt>
            </p>
            <p>
                This is an entity class used to encapsulate several data
                fields.  See <a href="txnexample_dpl.html#payloaddataentity">PayloadDataEntity.java</a> for
                implementation details.
            </p>
          </li>
        </ul>
      </div>
      <div class="sect2" lang="en" xml:lang="en">
        <div class="titlepage">
          <div>
            <div>
              <h3 class="title"><a id="txnguideexample_dpl"></a>TxnGuide.java</h3>
            </div>
          </div>
          <div></div>
        </div>
        <p>
            The main class in our example application is used to open and
            close our environment and store. It also spawns all the
            threads that we need. We start with the normal series
            of Java package and import statements, followed by our class
            declaration:
        </p>
        <pre class="programlisting">// File TxnGuideDPL.java

package persist.txn;

import com.sleepycat.db.DatabaseConfig;
import com.sleepycat.db.DatabaseException;
import com.sleepycat.db.DatabaseType;
import com.sleepycat.db.LockDetectMode;

import com.sleepycat.db.Environment;
import com.sleepycat.db.EnvironmentConfig;

import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.StoreConfig;

import java.io.File;
import java.io.FileNotFoundException;

public class TxnGuideDPL { </pre>
        <p>
    Next we declare our class' private data members. Mostly these are used
    for constants such as the name of the database that we are opening and
    the number of threads that we are spawning. However, we also declare
    our environment and database handles here.
</p>
        <pre class="programlisting">    private static String myEnvPath = "./";
    private static String storeName = "exampleStore";

// Handles
    private static EntityStore myStore = null;
                private static Environment myEnv = null;
    private static final int NUMTHREADS = 5; </pre>
        <p>
    Next, we implement our <tt class="function">usage()</tt> method. This
    application optionally accepts a single command line argument which is
    used to identify the environment home directory.
</p>
        <pre class="programlisting">    private static void usage() {
        System.out.println("TxnGuideDPL [-h &lt;env directory&gt;]");
        System.exit(-1);
    } </pre>
        <p>
    Now we implement our <tt class="function">main()</tt> method. This method
    simply calls the methods to parse the command line arguments and open
    the environment and store. It also creates and then joins the store writer
    threads.
</p>
        <pre class="programlisting">    public static void main(String args[]) {
        try {
            // Parse the arguments list
            parseArgs(args);
            // Open the environment and store
            openEnv();

// Start the threads
            StoreWriter[] threadArray;
            threadArray = new StoreWriter[NUMTHREADS];
            for (int i = 0; i &lt; NUMTHREADS; i++) {
                threadArray[i] = new StoreWriter(myEnv, myStore);
                threadArray[i].start();
            }

for (int i = 0; i &lt; NUMTHREADS; i++) {
                threadArray[i].join();
            }
        } catch (Exception e) {
            System.err.println("TxnGuideDPL: " + e.toString());
            e.printStackTrace();
        } finally {
            closeEnv();
        }
        System.out.println("All done.");
    } </pre>
        <p>
    Next we implement <tt class="function">openEnv()</tt>. This method is used
    to open the environment and then an entity store in that environment. Along
    the way, we make sure that every handle is free-threaded, and that the
    transactional subsystem is correctly initialized. Because this is a
    concurrent application, we also declare how we want deadlock detection
    to be performed. In this case, we use DB's internal block detector
    to determine whether a deadlock has occurred when a thread attempts to
    acquire a lock. We also indicate that we want the deadlocked thread
    with the <span class="emphasis"><em>youngest</em></span> lock to receive deadlock
    notification.
</p>
        <p>
    Notice that we also cause normal recovery to be run when we open the
    environment. This is the standard and recommended thing to do whenever
    you start up a transactional application.
</p>
        <p>
    Finally, notice that we open the database such that it supports
    uncommitted reads. We do this so that some cursor activity later in
    this example can read uncommitted data.  If we did not do this, then
    our <tt class="methodname">countObjects()</tt> method described later in
    this example would cause our thread to self-deadlock. This is because
    the cursor could not be opened to support uncommitted reads (that flag
    on the cursor open would, in fact, be silently ignored).
</p>
        <pre class="programlisting">    private static void openEnv() throws DatabaseException {
        System.out.println("opening env and store");

// Set up the environment.
        EnvironmentConfig myEnvConfig = new EnvironmentConfig();
        myEnvConfig.setAllowCreate(true);
        myEnvConfig.setInitializeCache(true);
        myEnvConfig.setInitializeLocking(true);
        myEnvConfig.setInitializeLogging(true);
        myEnvConfig.setRunRecovery(true);
        myEnvConfig.setTransactional(true);
        // EnvironmentConfig.setThreaded(true) is the default behavior 
        // in Java, so we do not have to do anything to cause the
        // environment handle to be free-threaded.

// Indicate that we want db to internally perform deadlock 
        // detection. Also indicate that the transaction that has
        // performed the least amount of write activity to
        // receive the deadlock notification, if any.
        myEnvConfig.setLockDetectMode(LockDetectMode.MINWRITE);

// Set up the entity store
        StoreConfig myStoreConfig = new StoreConfig();
        myStoreConfig.setAllowCreate(true);
        myStoreConfig.setTransactional(true);

// Need a DatabaseConfig object so as to set uncommitted read
        // support.
        DatabaseConfig myDbConfig = new DatabaseConfig();
        myDbConfig.setType(DatabaseType.BTREE);
        myDbConfig.setAllowCreate(true);
        myDbConfig.setTransactional(true);
        myDbConfig.setReadUncommitted(true);

try {
            // Open the environment
            myEnv = new Environment(new File(myEnvPath),    // Env home
                                    myEnvConfig);

// Open the store
            myStore = new EntityStore(myEnv, storeName, myStoreConfig);

// Set the DatabaseConfig object, so that the underlying
            // database is configured for uncommitted reads.
            myStore.setPrimaryConfig(PayloadDataEntity.class, myDbConfig);
        } catch (FileNotFoundException fnfe) {
            System.err.println("openEnv: " + fnfe.toString());
            System.exit(-1);
        }
    } </pre>
        <p>
    Finally, we implement the methods used to close our environment and
    databases, parse the command line arguments, and provide our class
    constructor. This is fairly standard code and it is mostly
    uninteresting from the perspective of this manual. We include it here
    purely for the purpose of completeness.
</p>
        <pre class="programlisting">    private static void closeEnv() {
        System.out.println("Closing env and store");
        if (myStore != null ) {
            try {
                myStore.close();
            } catch (DatabaseException e) {
                System.err.println("closeEnv: myStore: " +
                    e.toString());
                e.printStackTrace();
            }
        }

if (myEnv != null ) {
            try {
                myEnv.close();
            } catch (DatabaseException e) {
                System.err.println("closeEnv: " + e.toString());
                e.printStackTrace();
            }
        }
    }

private TxnGuideDPL() {}

private static void parseArgs(String args[]) {
        int nArgs = args.length;
        for(int i = 0; i &lt; args.length; ++i) {
            if (args[i].startsWith("-")) {
                switch(args[i].charAt(1)) {
                    case 'h':
                        if (i &lt; nArgs - 1) {
                            myEnvPath = new String(args[++i]);
                        }
                    break;
                    default:
                        usage();
                }
            }
        }
    }
} </pre>
      </div>
      <div class="sect2" lang="en" xml:lang="en">
        <div class="titlepage">
          <div>
            <div>
              <h3 class="title"><a id="payloaddataentity"></a>PayloadDataEntity.java</h3>
            </div>
          </div>
          <div></div>
        </div>
        <p>
    Before we show the implementation of the store writer thread, we
    need to show the class that we will be placing into the store. This
    class is fairly minimal. It simply allows you to store and retrieve an
    <tt class="literal">int</tt>, a <tt class="literal">String</tt>, and a
    <tt class="literal">double</tt>.  The <tt class="literal">int</tt> is our primary key.
</p>
        <pre class="programlisting">package persist.txn;
import com.sleepycat.persist.model.Entity;
import com.sleepycat.persist.model.PrimaryKey;
import static com.sleepycat.persist.model.Relationship.*;

@Entity
public class PayloadDataEntity {
    @PrimaryKey
    private int oID;

private String threadName;

private double doubleData;

PayloadDataEntity() {}

public double getDoubleData() { return doubleData; }
    public int getID() { return oID; }
    public String getThreadName() { return threadName; }

public void setDoubleData(double dd) { doubleData = dd; }
    public void setID(int id) { oID = id; }
    public void setThreadName(String tn) { threadName = tn; }
} </pre>
      </div>
      <div class="sect2" lang="en" xml:lang="en">
        <div class="titlepage">
          <div>
            <div>
              <h3 class="title"><a id="storewriter"></a>StoreWriter.java</h3>
            </div>
          </div>
          <div></div>
        </div>
        <p>
            <tt class="literal">StoreWriter.java</tt> provides the implementation
            for our entity store writer thread. It is responsible for: 
        </p>
        <div class="itemizedlist">
          <ul type="disc">
            <li>
              <p>
                    All transaction management.
                </p>
            </li>
            <li>
              <p>
                    Responding to deadlock exceptions.
                </p>
            </li>
            <li>
              <p>
                    Providing data to be stored in the entity store.
                </p>
            </li>
            <li>
              <p>
                    Writing the data to the store.
                </p>
            </li>
          </ul>
        </div>
        <p>
                In order to show off some of the ACID properties provided
                by DB's transactional support,
                <tt class="literal">StoreWriter.java</tt> does some things in a less
                efficient way than you would probably decide to use in a
                true production application. First, it groups 10 database
                writes together in a single transaction when you could just
                as easily perform one write for each transaction. If you
                did this, you could use auto commit for the individual
                database writes, which means your code would be slightly
                simpler and you would run a <span class="emphasis"><em>much</em></span>
                smaller chance of encountering blocked and deadlocked
                operations. However, by doing things this way, we are able
                to show transactional atomicity, as well as deadlock
                handling.
            </p>
        <p>
                To begin, we provide the usual package and import statements, and we declare our class:
            </p>
        <pre class="programlisting">package persist.txn;

import com.sleepycat.db.CursorConfig;
import com.sleepycat.db.DatabaseException;
import com.sleepycat.db.DeadlockException;
import com.sleepycat.db.Environment;
import com.sleepycat.db.Transaction;

import com.sleepycat.persist.EntityCursor;
import com.sleepycat.persist.EntityStore;
import com.sleepycat.persist.PrimaryIndex;

import java.util.Iterator;
import java.util.Random;
import java.io.UnsupportedEncodingException;

public class StoreWriter extends Thread
{ </pre>
        <p>
    Next we declare our private data members. Notice that we get handles
    for the environment and the entity store.  The random number generator 
    that we instantiate is used
    to generate unique data for storage in the database.  Finally, the
    <tt class="literal">MAX_RETRY</tt> variable is used to define how many times
    we will retry a transaction in the face of a deadlock. 
</p>
        <pre class="programlisting">    private EntityStore myStore = null;
    private Environment myEnv = null;
    private PrimaryIndex&lt;Integer,PayloadDataEntity&gt; pdKey;
    private Random generator = new Random();
    private boolean passTxn = false;

private static final int MAX_RETRY = 20; </pre>
        <p>
        Next we implement our class constructor. The most interesting thing about our constructor is
        that we use it to obtain our entity class's primary index.
    </p>
        <pre class="programlisting">    // Constructor. Get our handles from here
    StoreWriter(Environment env, EntityStore store)

throws DatabaseException {
        myStore = store;
        myEnv = env;

// Open the data accessor. This is used to store persistent
        // objects.
        pdKey = myStore.getPrimaryIndex(Integer.class,
                    PayloadDataEntity.class);
    } </pre>
        <p>
    Now we implement our thread's <tt class="methodname">run()</tt> method.
    This is the method that is run when <tt class="classname">StoreWriter</tt>
    threads are started in the main program (see <a href="txnexample_dpl.html#txnguideexample_dpl">TxnGuide.java</a>).
</p>
        <pre class="programlisting">    // Thread method that writes a series of records
    // to the database using transaction protection.
    // Deadlock handling is demonstrated here.
    public void run () { </pre>
        <p>
    The first thing we do is get a <tt class="literal">null</tt> transaction
    handle before going into our main loop. We also begin the top transaction loop here that causes our application to
    perform 50 transactions.
</p>
        <pre class="programlisting">        Transaction txn = null;

// Perform 50 transactions
        for (int i=0; i&lt;50; i++) { </pre>
        <p>
    Next we declare a <tt class="literal">retry</tt> variable. This is used to
    determine whether a deadlock should result in our retrying the
    operation. We also declare a  <tt class="literal">retry_count</tt> variable
    that is used to make sure we do not retry a transaction forever in the
    unlikely event that the thread is unable to ever get a necessary lock.
    (The only thing that might cause this is if some other thread dies
    while holding an important lock. This is the only code that we have to
    guard against that because the simplicity of this application makes it
    highly unlikely that it will ever occur.)
</p>
        <pre class="programlisting">           boolean retry = true;
           int retry_count = 0;
           // while loop is used for deadlock retries
           while (retry) { </pre>
        <p>
    Now we go into the <tt class="literal">try</tt> block that we use for
    deadlock detection. We also begin our transaction here.
</p>
        <pre class="programlisting">                // try block used for deadlock detection and
                // general exception handling
                try {

// Get a transaction
                    txn = myEnv.beginTransaction(null, null); </pre>
        <p>
        Now we write 10 objects under the transaction that we have just begun. 
        By combining multiple writes together under a single transaction,
        we increase the likelihood that a deadlock will occur. Normally,
        you want to reduce the potential for a deadlock and in this case
        the way to do that is to perform a single write per transaction. In
        other words, we <span class="emphasis"><em>should</em></span> be using auto commit to
        write to our database for this workload.
    </p>
        <p>
        However, we want to show deadlock handling and by performing
        multiple writes per transaction we can actually observe deadlocks
        occurring. We also want to underscore the idea that you can
        combing multiple database operations together in a single atomic
        unit of work. So for our example, we do the (slightly) wrong thing.
    </p>
        <pre class="programlisting">

// Write 10 PayloadDataEntity objects to the 
                    // store for each transaction
                    for (int j = 0; j &lt; 10; j++) {
                        // Instantiate an object
                        PayloadDataEntity pd = new PayloadDataEntity();

// Set the Object ID. This is used as the 
                        // primary key.
                        pd.setID(i + j);

// The thread name is used as a secondary key, and
                        // it is retrieved by this class's getName() 
                        // method.
                        pd.setThreadName(getName());

// The last bit of data that we use is a double
                        // that we generate randomly. This data is not
                        // indexed.
                        pd.setDoubleData(generator.nextDouble());

// Do the put
                        pdKey.put(txn, pd);
                    } </pre>
        <p>
        Having completed the inner database write loop, we could simply
        commit the transaction and continue on to the next block of 10
        writes. However, we want to first illustrate a few points about
        transactional processing so instead we call our
        <tt class="function">countObjects()</tt> method before calling the transaction
        commit. <tt class="function">countObjects()</tt> uses a cursor to read every
        object in the entity store and return a count of the number of objects
        that it found. 
    </p>
        <p>
        Because 
        <tt class="function">countObjects()</tt> 
        reads every object in the store, if used incorrectly the thread
        will self-deadlock.  The writer thread has just written 500 objects
        to the database, but because the transaction used for that write
        has not yet been committed, each of those 500 objects are still
        locked by the thread's transaction. If we then simply run a
        non-transactional cursor over the store from within the same
        thread that has locked those 500 objects, the cursor will
        block when it tries to read one of those transactional
        protected records. The thread immediately stops operation at that
        point while the cursor waits for the read lock it has
        requested.  Because that read lock will never be released (the thread
        can never make any forward progress), this represents a
        self-deadlock for the thread.
    </p>
        <p>
        There are three ways to prevent this self-deadlock:
    </p>
        <div class="orderedlist">
          <ol type="1">
            <li>
              <p>
                We can move the call to
                <tt class="function">countObjects()</tt> to a point after the
                thread's transaction has committed. 
            </p>
            </li>
            <li>
              <p>
                We can allow <tt class="function">countObjects()</tt> to
                operate under the same transaction as all of the writes
                were performed.
            </p>
            </li>
            <li>
              <p>
                We can reduce our isolation guarantee for the application
                by allowing uncommitted reads.
            </p>
            </li>
          </ol>
        </div>
        <p>
        For this example, we choose to use option 3 (uncommitted reads) to avoid
        the deadlock. This means that we have to open our underlying database such
        that it supports uncommitted reads, and we have to open our cursor handle
        so that it knows to perform uncommitted reads.
    </p>
        <pre class="programlisting">                    // commit
                    System.out.println(getName() + " : committing txn : " + i);
                    System.out.println(getName() + " : Found " +
                        countObjects(txn) + " objects in the store."); </pre>
        <p>
    Having performed this somewhat inelegant counting of the objects in the
    database, we can now commit the transaction.
</p>
        <pre class="programlisting">                    try {
                        txn.commit();
                        txn = null;
                    } catch (DatabaseException e) {
                        System.err.println("Error on txn commit: " +
                            e.toString());
                    }
                    retry = false; </pre>
        <p>
    If all goes well with the commit, we are done and we can move on to the
    next batch of 10 objects to add to the store. However, in the event
    of an error, we must handle our exceptions correctly. The first of
    these is a deadlock exception. In the event of a deadlock, we want to
    abort and retry the transaction, provided that we have not already
    exceeded our retry limit for this transaction.
</p>
        <pre class="programlisting">                } catch (DeadlockException de) {
                    System.out.println("################# " + getName() +
                        " : caught deadlock");
                    // retry if necessary
                    if (retry_count &lt; MAX_RETRY) {
                        System.err.println(getName() +
                            " : Retrying operation.");
                        retry = true;
                        retry_count++;
                    } else {
                        System.err.println(getName() +
                            " : out of retries. Giving up.");
                        retry = false;
                    } </pre>
        <p>
    In the event of a standard, non-specific database exception, we simply
    log the exception and then give up (the transaction is not retried).
</p>
        <pre class="programlisting">                } catch (DatabaseException e) {
                    // abort and don't retry
                    retry = false;
                    System.err.println(getName() +
                        " : caught exception: " + e.toString());
                    System.err.println(getName() +
                        " : errno: " + e.getErrno());
                    e.printStackTrace();  </pre>
        <p>
    And, finally, we always abort the transaction if the transaction handle
    is not null. Note that immediately after committing our transaction, we
    set the transaction handle to null to guard against aborting a
    transaction that has already been committed.
</p>
        <pre class="programlisting">                } finally {
                    if (txn != null) {
                        try {
                            txn.abort();
                        } catch (Exception e) {
                            System.err.println("Error aborting txn: " +
                                e.toString());
                            e.printStackTrace();
                        }
                    }
                }
            }
        }
    } </pre>
        <p>
    The final piece of our <tt class="classname">StoreWriter</tt> class is the
    <tt class="methodname">countObjects()</tt> implementation. Notice how in
    this example we open the cursor such that it performs uncommitted
    reads:
</p>
        <pre class="programlisting">    // A method that counts every object in the store.

private int countObjects(Transaction txn)  throws DatabaseException {
        int count = 0;

CursorConfig cc = new CursorConfig();
        // This is ignored if the store is not opened with uncommitted read
        // support.
        cc.setReadUncommitted(true);
        EntityCursor&lt;PayloadDataEntity&gt; cursor = pdKey.entities(txn, cc);

try {
            for (PayloadDataEntity pdi : cursor) {
                    count++;
            }
        } finally {
            if (cursor != null) {
                cursor.close();
            }
        }

return count;

}
} </pre>
      </div>
      <p>
    This completes our transactional example. If you would like to
    experiment with this code, you can find the example in the following
    location in your DB distribution:
</p>
      <pre class="programlisting"><span class="emphasis"><em>DB_INSTALL</em></span>/examples_java/src/persist/txn</pre>
    </div>
    <div class="navfooter">
      <hr />
      <table width="100%" summary="Navigation footer">
        <tr>
          <td width="40%" align="left"><a accesskey="p" href="txnexample_java.html">Prev</a> </td>
          <td width="20%" align="center">
            <a accesskey="u" href="wrapup.html">Up</a>
          </td>
          <td width="40%" align="right"> <a accesskey="n" href="inmem_txnexample_java.html">Next</a></td>
        </tr>
        <tr>
          <td width="40%" align="left" valign="top">Base API Transaction Example </td>
          <td width="20%" align="center">
            <a accesskey="h" href="index.html">Home</a>
          </td>
          <td width="40%" align="right" valign="top"> Base API In-Memory Transaction Example</td>
        </tr>
      </table>
    </div>
  </body>
</html>

Back to File Manager