Comments on: Of the impact of caching on your design (III) http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/ Notes on software development Tue, 06 Jan 2009 00:30:36 +0000 http://wordpress.org/?v=2.6.3 By: Denis http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-212 Denis Thu, 10 Jan 2008 21:02:48 +0000 http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-212 I am wary of the complexity that isolation levels would introduce. I you well know I am absolutely against confusing cache and database. In most cases I believe it is better structuring to use a context, although that context can be managed implicitely in a manner similar to how we manage transactions. This kind of freeze mechanism is very appealing to me. It avoids cloning. It is similar to isolation levels but it seems simpler to me (I may be wrong). I am wary of the complexity that isolation levels would introduce. I you well know I am absolutely against confusing cache and database.

In most cases I believe it is better structuring to use a context, although that context can be managed implicitely in a manner similar to how we manage transactions.

This kind of freeze mechanism is very appealing to me. It avoids cloning. It is similar to isolation levels but it seems simpler to me (I may be wrong).

]]> By: Hering Cheng http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-207 Hering Cheng Wed, 09 Jan 2008 19:38:21 +0000 http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-207 In many cases, the design outlined in the 3-part series is adequate where one always reads the "latest" data from cache. In other situations we /do/ want to hold on to "stale" data when reading the same data repeatedly from the cache. For example, when processing a base (or "prototype" or "template") quote from a market maker in an exchange, one of the initial validations (let's call it "A") may be to verify if the market maker is still "in the market" (e.g., if the market maker's computer has connectivity to the exchange). A later step of the processing (let's call it "B") of the quote (within the same transaction) may again attempt to look up that same market maker to retrieve the parameters of automatic quote generation at different price levels based on the quote template. In this case, we would want to use the same data at point "B" that was used at point "A". Now, for each example I can give, there will always be workarounds. For the specific example above, there are at least two workarounds: (1) All static/reference data can be collected -- and selectively cloned -- at the beginning of a transaction and passed on to latter part of the transaction as part of a "context". This is the approach I adopted at the exchange, mainly because it allows one to clearly delineate between computation and I/O -- a requirement when processing hundreds of thousands of transactions a second as many derivatives exchanges do. However, this is difficult to accomplish and requires careful structuring of code. (2) The data looked up at point "A" and point "B" can be separated into different caches and updated independently. Regardless of the workarounds, ideally a cache framework would give its users choices on isolation levels, similar to what contemporary RDBMSs do. In many cases, the design outlined in the 3-part series is adequate where one always reads the “latest” data from cache. In other situations we /do/ want to hold on to “stale” data when reading the same data repeatedly from the cache. For example, when processing a base (or “prototype” or “template”) quote from a market maker in an exchange, one of the initial validations (let’s call it “A”) may be to verify if the market maker is still “in the market” (e.g., if the market maker’s computer has connectivity to the exchange). A later step of the processing (let’s call it “B”) of the quote (within the same transaction) may again attempt to look up that same market maker to retrieve the parameters of automatic quote generation at different price levels based on the quote template. In this case, we would want to use the same data at point “B” that was used at point “A”.

Now, for each example I can give, there will always be workarounds. For the specific example above, there are at least two workarounds:

(1) All static/reference data can be collected — and selectively cloned — at the beginning of a transaction and passed on to latter part of the transaction as part of a “context”. This is the approach I adopted at the exchange, mainly because it allows one to clearly delineate between computation and I/O — a requirement when processing hundreds of thousands of transactions a second as many derivatives exchanges do. However, this is difficult to accomplish and requires careful structuring of code.

(2) The data looked up at point “A” and point “B” can be separated into different caches and updated independently.

Regardless of the workarounds, ideally a cache framework would give its users choices on isolation levels, similar to what contemporary RDBMSs do.

]]> By: Denis http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-202 Denis Tue, 08 Jan 2008 04:44:39 +0000 http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-202 Hering I agree with you that the dynamic proxies are more difficult to debug. This could be adressed in a number of ways but it is also the price of dynamism. As for the second part of your comments, can you explicit your use case ? It seems somewhat contradictory with my premise that you don't want to hold onto stale data when using a cache. Hering I agree with you that the dynamic proxies are more difficult to debug. This could be adressed in a number of ways but it is also the price of dynamism.

As for the second part of your comments, can you explicit your use case ? It seems somewhat contradictory with my premise that you don’t want to hold onto stale data when using a cache.

]]> By: Hering Cheng http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-188 Hering Cheng Wed, 02 Jan 2008 19:18:53 +0000 http://digitalbrikes.com/onebrikeatatime/2007/12/26/of-the-impact-of-caching-on-your-design-iii/#comment-188 In my previous job at the Pacific Exchange (now part of NYSE), we adopted an approach similar to the one proposed here, where application code interacts with a "proxy" to the actual cached object. We did not use Java dynamic proxy; instead, we used home-grown code generation technology where all of our domain objects are defined in XML files and a generator writes the Java classes. Our proxy also serves the double duty of providing the mutable interface; it merges the job of the proxy mentioned in Part II with that in Part III. We did not use Java dynamic proxy because it appeared to be more difficult to debug. At the company where Denis and I work we use Java dynamic proxy, and my experience is that it is difficult to track down where the actual logic of an accessor method resides. One issue that I'd like to see some comments on is how to deal with repeatable reads within a transaction. This is an issue for which we did not implement a generic solution at Pacific Exchange. Basically, we look up a piece of reference data (by primary key, say) at point X in a transaction and we read the same data again (using the same PK, say) at point Y within the same transaction. How do we make sure that we see exactly the same data at both points if that piece of reference data was modified and committed between times X and Y by another transaction? A naive solution is to pessimistically "clone" the data even for read-only purposes, at the cost of performance. Another is to make the transaction (or the "dependent" proxy) "remember" the version (of the reference data, or "independent", proxy) that was retrieved at point X and look up the same version at point Y. In my previous job at the Pacific Exchange (now part of NYSE), we adopted an approach similar to the one proposed here, where application code interacts with a “proxy” to the actual cached object. We did not use Java dynamic proxy; instead, we used home-grown code generation technology where all of our domain objects are defined in XML files and a generator writes the Java classes. Our proxy also serves the double duty of providing the mutable interface; it merges the job of the proxy mentioned in Part II with that in Part III. We did not use Java dynamic proxy because it appeared to be more difficult to debug. At the company where Denis and I work we use Java dynamic proxy, and my experience is that it is difficult to track down where the actual logic of an accessor method resides.

One issue that I’d like to see some comments on is how to deal with repeatable reads within a transaction. This is an issue for which we did not implement a generic solution at Pacific Exchange. Basically, we look up a piece of reference data (by primary key, say) at point X in a transaction and we read the same data again (using the same PK, say) at point Y within the same transaction. How do we make sure that we see exactly the same data at both points if that piece of reference data was modified and committed between times X and Y by another transaction? A naive solution is to pessimistically “clone” the data even for read-only purposes, at the cost of performance. Another is to make the transaction (or the “dependent” proxy) “remember” the version (of the reference data, or “independent”, proxy) that was retrieved at point X and look up the same version at point Y.

]]>