Domino hints and tips

Domino Performance Tuning

Lotus Notes performance tuning is complicated, because there are so many potential performance issues. Possible problem areas include the hardware, the operating system and application design. If the application design is really poor, it will be difficult to fix it by tuning the storage subsystem. If the processors or memory are underpowered, then this will affect the Indexer speed, the Replicator speed, the number of maximum possible database transactions, and the number of add-ins that can run in parallel. If the Network is overused, then this will slow down all transactions.
From the storage viewpoint, disk access rates and configurations affect the general database performance and performance of views and view indexes. So if you are attacking a problem with Notes performance from the storage angle, you need to consider, and eliminate potential problems outside the storage sphere also.

So where do you start? With the Network! Run a simple TELNET data transfer between your Lotus Notes Server and a client, and see what kind of speed you get, compared to what you would expect from your network. If your network is rated at 1GB/s you will be lucky to ever see that, but you should see speeds of about 500Mb/s or more. If you are seeing less that 500Mb/s then you probably have network problems. You really need a network expert to check this out, as so many things could be wrong.

If the Telnet speeds are Ok, then you need to check that your Domino systems are using the network efficiently, On Windows, check the Processor Object, then the Percent DPC (Deferred Process Calls) Time and DPCs Queued/Second. These will give you an idea of how much processor time is spent servicing the network. If the DPC queue is growing, then you have an escalating problem.

If the network is Ok, look at the performance of your Domino servers. If you have one Domino server, which is accessing one disk drive, then really all you can do is suggest that your developers tune their application. This is outside the scope of a storage website, so we will say no more.

If you have several disk drives, then you have some storage options. Domino is IO intensive. Make sure that the active workload is spread over all your available drives. Consider setting up disk drive partitions, and arranging the system, program, and data directories to distribute the workload across the drives as evenly as possible. You may have to experiment a bit to get this right. What you really need, is disk subsystems which have large amounts of cache, so all writes go to cache, and most reads come from cache.

back to top

Performance monitoring tools

Server.Load is supplied with the Domino install CD. It is GUI based and provides some basic performance analysis data. Its simple to use, but just provides basic statistics.

There are several good third party products which report on Domino performance. An example is RoboMon from Itheon. This runs on Windows NT or Windows 2000 servers. As well as looking for bottlenecks in the IO subsystem, it will also give you some recommendations on how to fix developing problems. It checks on database sizes, compared to preset thresholds, and warns if databases are growing at unexpected rates.

back to top

Undocumented problem debugging

This stuff is about changing the NOTES.INI file to get better debugging information about problems. If the commands are undocumented, then you should probably not be doing it without talking to Lotus Support. However, if you know what the options are, then you know what to discuss with them. If you decide to 'go it alone' and change NOTES.INI yourself, then please take a backup of your NOTES.INI file before you start, and also read the terms and conditions page that goes with this site, because you are on your own.
Its not a good idea to change the NOTES.INI file using a text editor, the best way is to change it using the NOTES.INI Settings tab in the Server Configuration document of the Domino Directory. So, now you are thoroughly frightened, lets see what options are available.

• By default, debugging information is sent to the server console. Use the parameter

    Debug_Outfile=filename
  

  to switch the trace to write to a file. That makes it much easier to analyze the trace afterwards. Its probably best to put this file into a separate partition, if you can, as its size is unpredictable. You don't want the trace log to fill up your main database partition. If the trace partition fills up, then Domino should stop tracing, but otherwise carry on processing as normal. The log file cannot be deleted while the Domino server is running; if an attempt is made to delete the file, a sharing violation will occur. If you specify a directory path for the trace file, then that path must exist. Domino will not create the path for you. Once the Debug_Outfile parameter has been added to the NOTES.INI file, the Domino server and the Server's Notes client must be shut down and restarted for logging to begin. it is important, however, to insert a carriage return after the last NOTES.INI entry.

• To list out AdminP schedule information, specify the parameter

   Debug_AdminP=1
  

  and you will see a schedule dump every time the schedule is updated. This needs Domino R5 to work.

• If you want to see the order in which documents in a database are replicated, use

   Debug_Repl_All=1
  

  The output looks like

  03-02-2003 15:33:29 AM REPLICA: Updating NoteID 0x206F in Roles.nsf
  03-02-2003 15:33:29 AM REPLICA: Deleting NoteID 0x205C in Roles.nsf
  

• If you want information on databases which are not replicating, use

   Debug_Repl_All=2 
    03-02-2003 05:31:00 AM REPLICA: ...Skipping note in Destination list 
   (UNID OCFFACF7EB:E2B4F2C4-ON672320AC:OC2347C9P3)
   03-02-2003 05:31:00 AM REPLICA: ...(Con't) 
   Destination has same or newer note. 
   Src Time/SeqNo=03-02-2003 05:31:00 AM
  

• Debug_AMGR is a setting that allows the Domino server to report Agent Manager (AMGR) activities. Several agent Manager flags can be set, here is just a few
  • '*' This means output everything! Its one to watch out for. It will need lots of disk space, and will degrade client performance a bit.
  • 'p' Generates Agent performance statistics
  • 'm' reports on Agent memory warnings

back to top

Storage Tips for Running Domino on OS/390

You can store Domino databases on OS/390 in two different file systems, VSAM HFS or zFS. If you use HFS, then you need to make sure that the files do not become full. Try to keep them at about 80% utilisation. DFSMS handles HFS file integrity on update, and can lock out access to the whole HFS file for one update. For that reason, its best to avoid large HFS files to minimise lockouts, and also avoid multi-volume files.

zFS has a better caching system. It detect sequential reads and pre-stage data in buffers to speed up performance. It also has a better lock/release mechanism. It is therefore much better to use zFS files for Domino databases.

The only major performance issue is channel utilisation, as Notes uses large block files. You need to monitor disk channel usage, and take action if it becomes excessive.

back to top

Copyright © Lascon Storage Ltd. 2000 to present date. By entering and using this site, you accept the conditions and limitations of use