Remote Auxiliary Cache Client / Server
The Remote Auxiliary Cache is an optional plug in for
JCS. It is intended for use in multi-tiered systems to
maintain cache consistency. It uses a highly reliable
RMI client server framework that currently allows for
any number of clients. Using a listener id allows
multiple clients running on the same machine to connect
to the remote cache server. All cache regions on one
client share a listener per auxiliary, but register
separately. This minimizes the number of connections
necessary and still avoids unnecessary updates for
regions that are not configured to use the remote cache.
Local remote cache clients connect to the remote cache
on a configurable port and register a listener to
receive cache update callbacks at a configurable port.
If there is an error connecting to the remote server or
if an error occurs in transmission, the client will
retry for a configurable number of tries before moving
into a failover-recovery mode. If failover servers are
configured the remote cache clients will try to register
with other failover servers in a sequential order. If a
connection is made, the client will broadcast all
relevant cache updates to the failover server while
trying periodically to reconnect with the primary
server. If there are no failovers configured the client
will move into a zombie mode while it tries to
re-establish the connection. By default, the cache
clients run in an optimistic mode and the failure of the
communication channel is detected by an attempted update
to the server. A pessimistic mode is configurable so
that the clients will engage in active status checks.
The remote cache server broadcasts updates to listeners
other than the originating source. If the remote cache
fails to propagate an update to a client, it will retry
for a configurable number of tries before de-registering
the client.
The cache hub communicates with a facade that implements
a zombie pattern (balking facade) to prevent blocking.
Puts and removals are queued and occur asynchronously in
the background. Get requests are synchronous and can
potentially block if there is a communication problem.
By default client updates are light weight. The client
listeners are configured to remove elements form the
local cache when there is a put order from the remote.
This allows the client memory store to control the
memory size algorithm from local usage, rather than
having the usage patterns dictated by the usage patterns
in the system at large.
When using a remote cache the local cache hub will
propagate elements in regions configured for the remote
cache if the element attributes specify that the item to
be cached can be sent remotely. By default there are no
remote restrictions on elements and the region will
dictate the behavior. The order of auxiliary requests is
dictated by the order in the configuration file. The
examples are configured to look in memory, then disk,
then remote caches. Most elements will only be retrieved
from the remote cache once, when they are not in memory
or disk and are first requested, or after they have been
invalidated.
Client Configuration
The configuration is fairly straightforward and is
done in the auxiliary cache section of the
cache.ccf
configuration file. In the example below, I created
a Remote Auxiliary Cache Client referenced by
RFailover
.
This auxiliary cache will use
localhost:1102
as its primary remote cache server and will attempt
to failover to
localhost:1103
if the primary is down.
Setting
RemoveUponRemotePut
to
false
would cause remote puts to be translated into put
requests to the client region. By default it is
true
, causing remote put requests to be issued as
removes at the client level. For groups the put
request functions slightly differently: the item
will be removed, since it is no longer valid in its
current form, but the list of group elements will be
updated. This way the client can maintain the
complete list of group elements without the burden
of storing all of the referenced elements. Session
distribution works in this half-lazy replication
mode.
Setting
GetOnly
to
true
would cause the remote cache client to stop
propagating updates to the remote server, while
continuing to get items from the remote store.
# Remote RMI Cache set up to failover
jcs.auxiliary.RFailover=
org.apache.commons.jcs3.auxiliary.remote.RemoteCacheFactory
jcs.auxiliary.RFailover.attributes=
org.apache.commons.jcs3.auxiliary.remote.RemoteCacheAttributes
jcs.auxiliary.RFailover.attributes.FailoverServers=
localhost:1102,localhost:1103
jcs.auxiliary.RFailover.attributes.RemoveUponRemotePut=true
jcs.auxiliary.RFailover.attributes.GetOnly=false
This cache region is setup to use a disk cache and
the remote cache configured above:
#Regions preconfigured for caching
jcs.region.testCache1=DC,RFailover
jcs.region.testCache1.cacheattributes=
org.apache.commons.jcs3.engine.CompositeCacheAttributes
jcs.region.testCache1.cacheattributes.MaxObjects=1000
jcs.region.testCache1.cacheattributes.MemoryCacheName=
org.apache.commons.jcs3.engine.memory.lru.LRUMemoryCache
Server Configuration
The remote cache configuration is growing. For now,
the configuration is done at the top of the
remote.cache.ccf
file. The
startRemoteCache
script passes the configuration file name to the
server when it starts up. The configuration
parameters below will create a remote cache server
that listens to port
1102
and performs call backs on the
jcs.remotecache.serverattributes.servicePort
, also specified as port
1102
.
# Registry used to register and provide the
# IRemoteCacheService service.
registry.host=localhost
registry.port=1102
# call back port to local caches.
jcs.remotecache.serverattributes.servicePort=1102
# rmi socket factory timeout
jcs.remotecache.serverattributes.rmiSocketFactoryTimeoutMillis=5000
# cluster setting
jcs.remotecache.serverattributes.LocalClusterConsistency=true
jcs.remotecache.serverattributes.AllowClusterGet=true
Remote servers can be chained (or clustered). This
allows gets from local caches to be distributed
between multiple remote servers. Since gets are the
most common operation for caches, remote server
chaining can help scale a caching solution.
The
LocalClusterConsistency
setting tells the remote cache server if it should
broadcast updates received from other cluster
servers to registered local caches.
The
AllowClusterGet
setting tells the remote cache server whether it
should allow the cache to look in non-local
auxiliaries for items if they are not present.
Basically, if the get request is not from a cluster
server, the cache will treat it as if it originated
locally. If the get request originated from a
cluster client, then the get will be restricted to
local (i.e. memory and disk) auxiliaries. Hence,
cluster gets can only go one server deep. They
cannot be chained. By default this setting is true.
To use remote server clustering, the remote cache
will have to be told what regions to cluster. The
configuration below will cluster all
non-preconfigured regions with
RCluster1
.
# sets the default aux value for any non configured caches
jcs.default=DC,RCluster1
jcs.default.cacheattributes=
org.apache.commons.jcs3.engine.CompositeCacheAttributes
jcs.default.cacheattributes.MaxObjects=1000
jcs.auxiliary.RCluster1=
org.apache.commons.jcs3.auxiliary.remote.RemoteCacheFactory
jcs.auxiliary.RCluster1.attributes=
org.apache.commons.jcs3.auxiliary.remote.RemoteCacheAttributes
jcs.auxiliary.RCluster1.attributes.RemoteTypeName=CLUSTER
jcs.auxiliary.RCluster1.attributes.RemoveUponRemotePut=false
jcs.auxiliary.RCluster1.attributes.ClusterServers=localhost:1103
jcs.auxiliary.RCluster1.attributes.GetOnly=false
RCluster1 is configured to talk to a remote server
at
localhost:1103
. Additional servers can be added in a comma
separated list.
If we startup another remote server listening to
port 1103, (ServerB) then we can have that server
talk to the server we have been configuring,
listening at 1102 (ServerA). This would allow us to
set some local caches to talk to ServerA and some to
talk to ServerB. The two remote servers will
broadcast all puts and removes between themselves,
and the get requests from local caches could be
divided. The local caches do not need to know
anything about the server chaining configuration,
unless you want to use a standby, or failover
server.
We could also use ServerB as a hot standby. This can
be done in two ways. You could have all local caches
point to ServerA as a primary and ServerB as a
secondary. Alternatively, you can set ServerA as the
primary for some local caches and ServerB for the
primary for some others.
The local cache configuration below uses ServerA as
a primary and ServerB as a backup. More than one
backup can be defined, but only one will be used at
a time. If the cache is connected to any server
except the primary, it will try to restore the
primary connection indefinitely, at 20 second
intervals.
# Remote RMI Cache set up to failover
jcs.auxiliary.RFailover=
org.apache.commons.jcs3.auxiliary.remote.RemoteCacheFactory
jcs.auxiliary.RFailover.attributes=
org.apache.commons.jcs3.auxiliary.remote.RemoteCacheAttributes
jcs.auxiliary.RFailover.attributes.FailoverServers=
localhost:1102,localhost:1103
jcs.auxiliary.RC.attributes.RemoveUponRemotePut=true
jcs.auxiliary.RFailover.attributes.GetOnly=false
Server Startup / Shutdown
It is highly recommended that you embed the Remote
Cache Server in a Servlet container such as Tomcat.
Running inside Tomcat allows you to use the
JCSAdmin.jsp page. It also takes care of the
complexity of creating working startup and shutdown
scripts.
JCS provides a convenient startup servlet for this
purpose. It will start the registry and bind the
JCS server to the registry. To use the startup
servlet, add the following to the web.xml file and
make sure you have the cache.ccf file in the
WEB-INF/classes directly of your war file.
<servlet>
<servlet-name>JCSRemoteCacheStartupServlet</servlet-name>
<servlet-class>
org.apache.commons.jcs3.auxiliary.remote.server.RemoteCacheStartupServlet
</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>
<servlet-mapping>
<servlet-name>JCSRemoteCacheStartupServlet</servlet-name>
<url-pattern>/jcs</url-pattern>
</servlet-mapping>
