OS X Server Caching

I knew since September that the server caching feature was coming to OS X and I have to say that I was excited to get my hands on it.

It is a great feature that offers bandwidth saving with no client configuration. As always it is better to configure one server than thousands of clients. Don’t you think?

For the caching server configuration the options are few. Basically you can:

  • Decide how much space you want to dedicate for the cache and on which hard drive you want to store it.
  • Enable or disable the caching server
  • Delete all cached updates and apps

This is what the server does automatically without any visual clue for you:

  • The server launches the AssetCache process as the _assetcache user
  • A LaunchDaemon is created for the service
  • The cache server registers online with Apple and provides it’s public IP, your servers local IP, internal DNS name? (not sure of the dns)
  • It gets in return a validation key to prove to client computers its authenticity
  • Creates the folder /Library/Server/Caching/Data to store the cached downloads

When a client tries to download an app or download a software update:

  • Contacts Apple
  • Apple checks if there is any caching server registered for that public IP
  • If there is one Apple returns to the client the details of the caching server
  • The client computer contacts the caching server and requests the update or app
  • The server identifies itself as a valid/trusted source with the validation key
  • The caching server downloads the app or update if it doesn’t have it, and then serves it to the client computer

You should know:

  • For now only 10.8.2 clients get any benefit of this.
  • iOS devices do not benefit from this service
  • The caching server caches both App Store purchases and software updates
  • If a client computer is pointing manually to a custom SUS the caching server is ignored (does this also affect app purchases?)
  • The SUS service and the caching service do not share their data. You might end up with two copies of all the updates
  • There is no need for you to re-direct connections using DNS trick anymore
  • If you are not directly configuring clients to point to your local SUS but using DNS tricks. You will probably end up deleting the local SUS and cleaning the DNS table. The caching server is for you!
  • Evey 55 minutes your caching server will contact Apple to verify the public IP still the same and get a new token (see next log below)
2012/12/09 19:57:39:041  Request for registration from https://lcdn-registration.apple.com/lcdn/register failed: HTTP response 401, body "REQUEST_REAUTH"; retrying with new token
2012/12/09 19:57:39:317  Request for registration from https://lcdn-registration.apple.com/lcdn/register succeeded
2012/12/09 19:57:39:317  Got back public IP xxx.xxx.xxx.xx

There is not much documentation available at the moment. Here you have how the control pane, and by clicking it you’ll download the help page

 

In case you are wondering, the real log file does not seem to be available from the Server.app for some reason. The server app offers a grep of the system.log instead? bug?

In any case the log file is located at /Library/Server/Caching/Logs/Debug.log and here is a truncated output of first a successful server download for a client and second the server offering an update that was cached already (0 downloaded)

bash-3.2# cat /Library/Server/Caching/Logs/Debug.log
...
2012/12/10 19:32:46:427  ECResponse[0x7fdc1a41c060]: Got request for host = http://swcdn.apple.com/content/downloads/49/01/041-9115/6o56uk1ef7ejpe0rwzox35nad49tq039ir/iWork_9.3_Update.pkg
2012/12/10 19:32:46:428  ECAssetHandler[0x7fdc1a4173b0]: Initializing asset handler for http://swcdn.apple.com/content/downloads/49/01/041-9115/6o56uk1ef7ejpe0rwzox35nad49tq039ir/iWork_9.3_Update.pkg (path = /Library/Server/Caching/Data/FD4F6203-345D-43C9-B7A8-0DD3D12085E5)
2012/12/10 19:32:46:428  ECAssetHandler[0x7fdc1a4173b0]: Cached asset length = 399791345 MD5=<3ac79d7e 87d4488f 3e26ff01 cb1dbfc3> last modified Tue, 04 Dec 2012 02:54:54 GMT
2012/12/10 19:32:46:429  ECAssetHandler[0x7fdc1a4173b0]: Extents loaded from disk: [0,399791345]
2012/12/10 19:32:46:429  ECAssetHandler[0x7fdc1a4173b0]: Request had no range headers
2012/12/10 19:32:46:429  ECAssetRequestor[0x7fdc1b018030]: Data already cached for asset http://swcdn.apple.com/content/downloads/49/01/041-9115/6o56uk1ef7ejpe0rwzox35nad49tq039ir/iWork_9.3_Update.pkg, issuing If-Modified-Since request
2012/12/10 19:32:47:093  ECResponse[0x7fdc1a41c060]: Info loaded: file length = 399791345, reader = 0x7fdc1a41aed0
2012/12/10 19:32:47:094  ECCacheReader[0x7fdc1a41aed0]: fetchDataOfLength called with file closed at offset 0
2012/12/10 19:32:47:094  ECAssetHandler[0x7fdc1a4173b0]: Opened extent [0, 399791345] for reading at offset 0 (offset into file = 0)
2012/12/10 19:32:47:094  ECCacheReader[0x7fdc1a41aed0]: opened offset [0, 399791345]
2012/12/10 19:32:47:095  ECAssetRequestor[0x7fdc1b018030]: Outgoing connection finished
...
2012/12/10 19:35:44:342  ECResponse[0x7fdc1a41c740]: Got request for host = http://swcdn.apple.com/content/downloads/07/07/041-8970/ta2kc1upzafr99hm7igkqq02hi3lwaoir7/ChineseWordlistUpdate.pkg
2012/12/10 19:35:44:344  ECAssetHandler[0x7fdc1a46c920]: Initializing asset handler for http://swcdn.apple.com/content/downloads/07/07/041-8970/ta2kc1upzafr99hm7igkqq02hi3lwaoir7/ChineseWordlistUpdate.pkg (path = /Library/Server/Caching/Data/5128AF14-D64B-4865-A76C-66BD571BA2F2)
2012/12/10 19:35:44:344  ECAssetHandler[0x7fdc1a46c920]: Cached asset length = 47543 MD5=<805a210f 4129352c 69fdcea0 05345dbe> last modified Mon, 03 Dec 2012 18:36:03 GMT
2012/12/10 19:35:44:344  ECAssetHandler[0x7fdc1a46c920]: Extents loaded from disk: [0,47543]
2012/12/10 19:35:44:344  ECAssetHandler[0x7fdc1a46c920]: Request had no range headers
2012/12/10 19:35:44:344  ECAssetRequestor[0x7fdc1dd02f00]: Data already cached for asset http://swcdn.apple.com/content/downloads/07/07/041-8970/ta2kc1upzafr99hm7igkqq02hi3lwaoir7/ChineseWordlistUpdate.pkg, issuing If-Modified-Since request
2012/12/10 19:35:45:282  ECResponse[0x7fdc1a41c740]: Info loaded: file length = 47543, reader = 0x7fdc1a46f660
2012/12/10 19:35:45:282  ECCacheReader[0x7fdc1a46f660]: fetchDataOfLength called with file closed at offset 0
2012/12/10 19:35:45:282  ECAssetHandler[0x7fdc1a46c920]: Opened extent [0, 47543] for reading at offset 0 (offset into file = 0)
2012/12/10 19:35:45:282  ECCacheReader[0x7fdc1a46f660]: opened offset [0, 47543]
2012/12/10 19:35:45:282  47543 bytes served, 47543 from cache, 0 downloaded
2012/12/10 19:35:45:282  ECCacheReader[0x7fdc1a46f660]: canceled at offset = 47543
2012/12/10 19:35:45:282  ECAssetHandler[0x7fdc1a46c920]: Asset has no clients: http://swcdn.apple.com/content/downloads/07/07/041-8970/ta2kc1upzafr99hm7igkqq02hi3lwaoir7/ChineseWordlistUpdate.pkg
2012/12/10 19:35:45:282  ECAssetHandler[0x7fdc1a46c920]: Removed reader 0x7fdc1a46f660, will cancel 0 requestors
2012/12/10 19:35:45:282  Total bytes returned to clients 2786669731, total bytes requested from servers 388568166
2012/12/10 19:35:45:283  ECAssetRequestor[0x7fdc1dd02f00]: Outgoing connection finished

 

 

Posted in IT and stuff Tagged with: ,
26 comments on “OS X Server Caching
  1. ionfref says:

    I’m a little unclear how a client is matched up with a caching server.

    Is the caching limited to the subnet that the server is on?

    • nbalonso says:

      It uses the public IP.

      As long as your caching server and your client computers connect to the internet using the same public IP, Apple will take care of introducing the server to the clients.

      (The clients being able to contact your server directly is implied)

      • Maurits says:

        I tested the following setup:
        Internet – ADSLNATrouter – LAN1 – NATrouter2 – LAN2

        LAN1 has 172.16.3,xx net
        LAN2 has 192.168.1.xxx net

        There are several separate LAN’s (LAN2 etc) that connect to the internet using an office NATrouter.
        I connected the caching servicer to LAN1 (so IP 172.16.3.x) and a client in LAN2 (has different IP address in 192.168.1.x range but same internet facing IP address (of the ADSLNATrouter)) and this client uses the caching service ! great.

        I have to do a ethernet dump, but the smart thing is the software update, the responce from Apple will be based on your clients internet facing IPv4 address and the responce will be:
        -contact your local Caching Server at IP…
        -if that fails, use Akamai.

        This can be done since the Caching Service registers itself with Apple.
        Some documentation is in the help of server.app

        • nbalonso says:

          that is exactly the expected result.

          In my case because our clients are “all” pointing to an internal SUS, my cache size still under 2GB. Either our clients are not purchasing many apps or the service needs some tweaking.

          I’d be interested in knowing what you see with tcpdump or Wireshark. If you could post the filters you use that will help troubleshooting.

          PS: the help documentation is also available here when you click on the panel image

      • Peter says:

        Do the clients need to be bound in OD to the server at all? Or truly just connecting to the internet from the same public IP address.

  2. Chris says:

    we have two external (public) IP addresses serving the network for redundancy and load balancing – how will this scenario play out? Can I somehow register both public IP addresses with Apple for the one server?

    • nbalonso says:

      It’s hard to say without any official documentation, but I really doubt that you will get any more control.

      The caching server does an online check every 55 minutes, maybe if the external IP changes the server will register again.

      One positive point on this is that the cached data persists between registrations

      • Chris says:

        Looking at the debug log it only ever seems to successfully register with one of the public IP addresses. There are a few failures in there – I wonder if they are seen as coming from the other public address and somehow the server ID and public address don’t match the initial registration and so are rejected?

        I guess any client requests from the ‘wrong’ IP will not be rerouted to the local server for updates so it’s probably a ~50% success I can expect.

        • Chris says:

          Looks like that’s the way it’s working from initial trials. Need to configure outbound update requests to use the one public link somehow to get the best out of this.

        • Jon Jewett says:

          You could try running two instances of the caching service (two servers) or better yet maybe just multi home the caching server with two static LAN IP’s, and then write NAT rules that would ALWAYS direct all outbound traffic from server IP one over internet IP one, and server IP two over Internet IP two. This should allow the registration of both internet IPs, and regardless of which internet IP a client was using they’d be directed to a LAN caching server without issue.

          (mind you this was simply an idea, and it seems to me like it should work fine, I have not actually done it).

  3. Richard Smith says:

    I’m interested in seeing how it might work in an environment that isn’t NAT’d (ie both server and client have their own public IP’s)

  4. Jim says:

    I tried this on a multi-homed server; one leg in public IP space, the other in 10.x.y.z space. It said something like, “caching server cannot be used on a public IP address.” So, looks like the caching server itself can’t be directly attached the public Internet but probably has to be on a NAT’d subnet.

  5. Marc K says:

    Not sure if I’m understanding this correctly… This also handles Apple Software Updates? Does this eliminate the need for my SUS?

    • nbalonso says:

      If you are using a SUS just to save bandwith yes, no need for it. But make sure you delete the configuration on your clients that points them to the local SUS. PDF reads “…Also, if you
      configure your client to use the Software Update server, it takes precedence and the client
      cannot use the Caching server…”

      But if you are using a local SUS to control what updates your clients get, then no, stick with your local SUS.

  6. Mike Kingsley says:

    I would want to know more on like you mentioned does it serve up app updates even if you have a SUS? But that’s not essentially important to me.

    The line you mention:
    The caching server downloads the app or update if it doesn’t have it, and serves it to the client computer (could not yet verify this line is accurate)

    It seems important for us to figure that one out.

    Last if clients are on a different subnet I see it works but using what ports? (in case our network guy is doing a lot of blocking between subnets)

  7. Mike Kingsley says:

    Actually I just say this for advanced settings for caching service. At least it looks like you can set a port if you need. What would it pick otherwise?

    http://support.apple.com/kb/HT5590?utm_source=dlvr.it&utm_medium=twitter

  8. Kevin Neal says:

    I’m unable to turn on the caching service, I get an error saying the ListenRanges is invalid, I suspect this service is not available if you are using Link Aggregation on the Server, can any one confirm?

    • bob says:

      I get the same error when using VLANs on the server.
      Not quite ready for prime time would be my comment.

    • Eric McMurry says:

      Link aggregation broke this for me as well. Adding the bonded interface to the config file (/Library/Server/Caching/Config/Config.plist) brought it back.

      Try adding this:

      Interface
      bind0

      (replace bind0 with whatever your lag if name is).

  9. jon says:

    This service seems completely unreliable. I tried a few 25MB downloads and it worked fine. Then purchased Aperture, 600MB, and the cache didn’t increase. Same with Final Cut Pro. This is after a clean install with nothing else added. The reason for the clean install is that it previously stopped at 6.04GB and failed to cache anything after that.

  10. Valberto Cavalcante says:

    I changed my Private IP (change router) from 192.168.0.xxx to 192.168.100.xxx, after this the caching stopped working. What I can do? Thanks.

    • win1dmin says:

      This is sooo stupid – what sane enterprise would have NATed connections? Really? Are we living in dialup era?

      • jlg89 says:

        Not sure what planet you’re from, but here on planet Earth, almost every home and most businesses, even some rather large enterprises like Apple and Microsoft, use NAT for all or part of their network. NAT has nothing to do with dialup, it has to do with “stretching” the IPv4 address space until we can all make the switch to IPv6.

        In fact, though, the Caching Service doesn’t care whether or not it’s behind a NAT. The edge router has a public IP either way, and that’s what Apple’s service uses to determine what subnet(s) are served by a given caching server. The Caching Service works, NAT or not.

  11. Marc says:

    How do I track usage of the Caching service to ensure it’s operating properly?

  12. Rkahle says:

    We have the app store blocked for dynamic ips, with a cache server is it possible to update apps dynamically since you are updating from the cache server and not the app store?

4 Pings/Trackbacks for "OS X Server Caching"
  1. [...] things, but what exactly and how? Get a first look at this new service over at Noel Balonso’s blog. Source: nbalonso.com About Nate WalckNate is a Macintosh Systems Engineer at Tamman Technologies, [...]

  2. [...] Blog von Noel Alonso beschreibt detaillierter, was es damit auf sich hat. [...]

  3. [...] The Caching Service is a deceptively magical new service which automatically caches Apple update and Mac App Store content with no need for client configuration. A very good writeup of what the service actually does can be found at Noel Alonso’s blog. [...]

  4. Apple-Maroc.com : Le Premier Site Apple Au Maroc – Zoom sur la mise en cache d’OS X Server says:

    [...] B. Alonso explique par exemple que lors de la mise en place du cache, le serveur communique à Apple son adresse IP [...]

Leave a Reply