This entry is an excerpt of a report prepared for an enterprise client of IBM Lotus Quickr, running several clusters in staging, development and production. The project was completed a year ago and the client has seen many benefits to enabling DAOS. However, as the documentation on DAOS specific to Quickr is nonexistent, we learned to take into account several other factors…
Introduction
Domino Attachment Object Store is a low-level option on Domino servers (beginning with version 8.5) that allows files to be saved to the file system and referenced seamlessly as if they were still in the Domino nsf structure. Two advantages of DAOS in a traditional environment are lower disk space usage and improved backup time.
In a Quickr environment, not much disk will be saved overall, as most of the attachments are different and de-duplication of files is the primary reason for this savings. However, as most attachments are stored on the file system and only changed when the attachment is changed from the web UI, incremental backups will be significantly faster due to the much smaller size of nsfs and <1% of files changed in a day.
My experience from an enterprise client with fourteen Quickr servers has been phenomenal savings in backup time – more than 85% reduction from what used to be approaching a 24 hour cycle across production.
When DAOS is enabled, attachments over a specified size are saved to disk as “nlo” files. These are the attachments and are – speaking in binary terms – identical to their more friendly-named files such as Word, PDF, Excel documents. “nlo” stands for Notes Linked Object.
It is important to note that DAOS operates on a single server level. Normally this means that nlo files are encrypted with a specific server’s ID, and not shared among clusters. If this is the case, a place cannot be moved from one server to another when there are DAOS files detached from the nsf. However, in any Quickr environment, I recommend that we disable encryption of the attachments on the file system by implementing the notes.ini variable “DAOS_Encrypt_NLO=0”. This must be done on each server prior to enabling DAOS. This will allow us to reconstitute a Quickr place on another server for disaster recovery, testing, development, etc.
Quickr Templates manual step (pre-DAOS)
Quickr databases are based on templates stored in the directory <domino data>\LotusQuickr\Areatypes. When DAOS is first enabled, the PlaceTypes have DAOS turned on. However, some of the Quickr “.ntf” files need to have this turned on manually. To do so, the database property should be checked: “Use Domino Attachment Object Store” on the databases PageLibrary.ntf and MeetingRoom.ntf.
Moving a Place
In order to move a place temporarily from one server to another for testing, we must “un-DAOS” the place. This activity puts all of the attachments back into the place. Testing has revealed that although not specifically documented, this work on both a single NSF and on a directory full of them – which is how Quickr places are organized.
To prepare a place for moving, the following command should be initiated from Domino Administrator, RECON, the Domino server console, or from the Java console:
>load compact –C –daos off lotusquickr\placename
This will a) turn off DAOS on the database properties and b) reattach the attachments with their original names. Note that the NLO attachments will remain on the file system for a default 30 days unless they are pruned.
After performing this command, it is important to review the log.nsf file (or if the place has few rooms, the console) to determine if there were any errors during the compact operation. One error that was seen once during testing, actually happened while turning DAOS on and relates to attachments failing to detach – hence not an impact in this case.
After the log is reviewed, the place may be archived (qptool archive), or the server downed and the place copied to its new location.
Once in its new location, if the target server has DAOS enabled, the place may be “re-DAOS” enabled by using the compact command again with the place directory as an argument:
>load compact –C –daos on lotusquickr\placename
Performance Notes and Caveats
It is not recommended by IBM that the DAOS base path is on the same drive as the Domino Data directory, for disk performance reasons. However, this recommendation is based on the preponderance of Domino MAIL servers using DAOS and the high transaction rate of mail servers. In our case, Quickr attachments will be detached once, then a miniscule percentage of changes made to them in a given day. That coupled with the disk space available on other volumes often being scarce, and the significant backup benefits, there is enough reason to not heed this recommendation.
It should be noted for the future that transaction logging, which is on, is a prerequisite for DAOS, so should not be turned off while DAOS is enabled.
At Lotusphere 2012, it was recommended that we do NOT decrease the minimum object size to 4000 bytes as sometimes discussed, but for management purposes and performance of the DAOS catalog keep it at the higher 64000 byte level. This means we will also only require one pass per server for DAOS enablement.
This should be enough to get you started. In addition to the above, we conducted numerous tests and published results, developed a Production Plan, identified five Scenarios for disaster recovery and restoration procedures, and built a DAOS utility to gather NLO file references that was a necessity for Quickr. If you are interested in this additional analysis and how it may apply to your situation, please let me know in the comments. I can’t send it out as is, but am happy to answer questions you have about DAOS in general or Quickr in specific!
This week is a week of announcements in our tech world (check the at wires 3AM EDT tomorrow, for instance…). My own announcement is that SNAPPS has become a partner with a new major vendor, Intuit, with their QuickBase online database SaaS solution. This effort is 8 months in the making, and I’m very pleased to say that after several visits with Intuit QuickBase folks both at their headquarters and other events, I’m very excited about the prospects of working together on the next generation of online collaboration — the generation that takes pure generic collaboration and easily extends it to very specific, functional areas of business. As well, I look forward to creating some interesting integrations with IBM software.
