I am busy at the moment writing various implementations of NeuClear Ledger in mind with the next release 0.4 which should be released later this week. One of the the implementation plugins will be a relatively complete version that I’ve written in Prevayler

What are Ledgers?

Within NeuClear we use ledger’s as one of the most important building blocks. Ledger’s are the fundamental datastructure used in any kind of double entry accounting system.

The NeuClear Ledger API is fairly simple and can be used to implement ledgers using SQL, in memory etc

The API basically allows you to perform accounting transactions on the ledger implementation. It will then allow you to view balanaces etc.

The NeuClear system has a special requirement for what we called Held Transactions, which are simply temporary transactions that put a hold on the available funds of a system. It was important that we implemented this on the ledger side and not higher up in the NeuClear framework.

Our Requirements for a ledger for a transaction processor

The Prevalent Ledger is intended to be run on our NeuClear Transaction Processor. The Transaction Processor does the following:

1. Listens to Transfer Requests
2. Verifies the Signatures of said Requests
3. Verifies the Balance of Transferer
4. Performs book entry transaction on Ledger
5. Signs and returns a receipt

For performance and security reasons this processor will not allow people to view statements or check for historical records. This will be handled on external Auditing Servers, who use a different implementation of the NeuClear Ledger Hibernate

Thus the Prevalent model works fine. We have a few very limited types of queries we need to do in the normal run of things and require absolute speed and reliability.

30 second overview of Prevayler

Prevayler stores the entire database in memory and performs periodic snapshots to disk in a serialized form. To avoid dataloss and manage transactions any transactions are handled in serialized command objects. That get serialized to a transaction log on disk before they get executed.

For more info see the 1 Minute Introduction to Prevayler or the 2 minute tutorial both at on the Codehaus Wiki.

Implementing the Prevalent Ledger

A prevayler database needs a single reference point for all the data stored within. In Prevayler terminology this is the Prevalent System. In my implementation I created a central class LedgerSystem, which

The Data Objects

The most basic part of the ledger functionality is to get the balances of books and add transactions. In this implementation we keep no history just update the balances. To do this I created a super simple class BalanceTable which allows me to get the balance of a book as well as add an amount to a book.

For memory and speed reasons I chose to use TObjectDoubleHashMap from Trove4j as it stores primitive doubles and supposedly uses a lot less memory.

I wont get into the other parts of LedgerSystem here, but they are also fairly simple. The important part is to remember that the system is your main database and it needs to be memory efficient as well as Serializable. I made most of the methods in the System as well as its contained objects package scope, to not allow any outside manipulation of data.

The Commands

To create a Transfer I simply created a new command class which implements “TransactionWithQuery”. This simple one is called PostTransaction. All the command classes must be serializable. I pass all parameters to it’s constructor. The real juice of the object is done in the executeAndQuery() method which gets passed an execution time and the System.

public Object executeAndQuery(Object prevalentSystem, Date executionTime) throws Exception {
         LedgerSystem system=(LedgerSystem) prevalentSystem;
         TransactionTable table=system.getTransactionTable();
         if (table.exists(tran.getId()))
             throw new TransactionExistsException(null,tran.getId());
         if (table.exists(tran.getRequestId()))
             throw new TransactionExistsException(null,tran.getRequestId());
 
         table.register(tran.getId(),executionTime);
         table.register(tran.getRequestId(),executionTime);
 
         Iterator iter=tran.getItems();
         while (iter.hasNext()) {
              TransactionItem item = (TransactionItem) iter.next();
              system.getBalanceTable().add(item.getBook(),item.getAmount());
          }
 
         return new PostedTransaction(tran,executionTime);
     }

As you can see it’s pretty simple and straightforward. No real black magic. When you execute the above command, Prevayler first stores the serialized form to disk as its log and then executes the object.

Integrating it

All implementations of NeuClear Ledger must extend the Abstract class Ledger. This I did in PrevalentLedger.

The real juice here is the Constructor which loads the data:

    public PrevalentLedger(final String id, final String basedir) throws
		 IOException, ClassNotFoundException {
         super(id);
         prevayler = PrevaylerFactory.createPrevayler(new LedgerSystem(id), basedir);
         system=(LedgerSystem) prevayler.prevalentSystem();
     }

basedir is the directory used to store the snapshots and transaction logs. The new LedgerSystem created in createPrevayler is only used if there isn’t already one available as a snapshot in the basedir.

The Ledger class has various abstract methods that I needed to implement:

   public PostedTransaction performTransaction(UnPostedTransaction trans) 
	throws UnBalancedTransactionException, LowlevelLedgerException, InvalidTransactionException {
         try {
              return (PostedTransaction) prevayler.execute(new PostTransaction(trans));
          } catch (Exception e) {
              if (e instanceof InvalidTransactionException)
                  throw (InvalidTransactionException)e;
              if (e instanceof UnBalancedTransactionException)
                  throw (UnBalancedTransactionException)e;
              if (e instanceof LowlevelLedgerException)
                  throw (LowlevelLedgerException)e;
              throw new LowlevelLedgerException(e);
          }
     }

This creates a new Instance of PostTransaction and lets Prevayler execute it. Note the ugly Exception handling. This is due to the fact that the executeWithQuery method is declared with throws Exception. I’m sure there is a more elegant way, but I couldnt get my head around it at the moment.

Conclusion

It was remarkably easy to implement the Prevalent Ledger. I wrote it to the stage it is today in probably less than 8 hours over a 2 day period. Very refreshing.

For some things the logic got a tiny bit out of hand. See for example HoldTable and it’s pal AccountHeld this would have been very quick to do in SQL. I’m sure there is a better way here, I’ll have to look at it another time.

I guess my main fear about using Prevayler is when I make database changes, how easy will it migrate? I have heard there are various XML backends for Prevayler, which may suit this purpose well.

I am now working on a new implementation of Ledger, written using Hibernate. There is a bit more of a learning curve as I havent used Hibernate before, but I can see that it is an excellent tool.