Handle.Net Software Version 7 Release Notes & Change Logs

Version 7.0 of the Handle System constituted, released in December 2010, was a major upgrade to the system. The current release is version 7.3.1.

Template handles: a single template handle can be included in the form of a base formula that allows any number of extensions to that base to be resolved as full handles, according to a pattern, without each such handle being individually registered. This would allow, for example, the use of handles to reference an unlimited number of ranges within a video without each potential range having to be registered with a separate handle. If the pattern needs to be changed, e.g., the video moves or a different kind of server is used to deliver the video clips, only the single base handle needs to be changed to allow an unlimited number of previously constructed extensions to continue to resolve properly.

Offline signatures: handle values can be signed with "offline" private keys (private keys that need not exist on any Internet-connected machine). This additional layer of verification was applied to all entries in the Global Handle Registry by CNRI but any party that has the authority to create handle records at any level can use this general capability. The release includes a simple but flexible API for building handle value digests and signing those digests.

Multiple location type: support was added for a new handle value type, 10320/loc, which specifies a list of URL locations, including information that differentiates the locations, to which a handle can resolve. The web-to-handle proxy and Firefox extension can use the data that comes along with the http request, e.g., client IP address, browser language, URL parameters, etc) to determine the selection of a specific URL from the list of alternatives. Thus, European users could be directed to one URL while North American users could be directed to another.

DNS interface: added a DNS interface to translate DNS resolution requests to handle resolution requests. This includes support for translating DNS names to handles, including decoding Internationalized Domain Names.

Berkeley database: the distribution includes the Berkeley DB, which is the default storage system for version 7 of the Handle System. However, other databases may be used in place of the Berkeley DB, if desired.

Jython interpreter: added a Jython interpreter that can run script files or show an interactive shell, enabling any Python programmer to create scripts to interact with the handle server. Command-line parameters can specify authentication information or prompt the user with a GUI authentication window.

The Change Log below provides further detail about these changes, and a number of other changes and minor bug fixes.

 

Change Log - Version 7.3.1

  1. Prevented a multi-threaded client from attempting to simultaneously authenticate a single session multiple times, which led to authentication failure messages from the server.
  2. Added hdl-keygen tool to generate RSA and DSA keypairs for use by HANDLE.NET tools.
  3. Increased default backlog on server TCP interfaces to 50.
  4. Added additional configuration keys for SQL storage: "store_handle_as_string", "store_na_as_string", and "store_handle_value_type_as_string". These allow the relevant values to be stored as strings rather than as bytes in the backend storage. Most users should not need this, but it may be helpful for some users of an Oracle SQL backend.
  5. Enabled use of RSA keys for servers. Note: since some older clients can not check RSA signatures, we still recommend the use of DSA keys for now. DSA is used by default during server setup.
 

Change Log - Version 7.3

  1. Corrected the order of parameters for AdminRecord constructor in the client library.
  2. Fixed encoding of admin records so that they don't include extra bytes.
  3. Atomic file writing operations now used to preserve file backups on errors, such as out of disk space.
  4. Client now uses sessions by default; turn sessions off with resolver.setSessionTracker(null).
  5. Session Diffie-Hellman keys are now initialized on first use rather than on construction to prevent performance hit from turning sessions on by default.
  6. Added more detail in error messages when exceptions are thrown in the server.
  7. Signature code now uses algorithm DSAwithSHA1 instead of just DSA.
  8. After processing create-handle and add-values requests with overwrite-when-existing flag set, server indicates in response whether the given handle or values already existed, by using the overwrite-when-exists flag in the response.
  9. Improved the default choices on hdl-setup-server such as turning logging on.
  10. Prevent errors with refreshing replication sites at handle server start-up from causing start-up to fail.
  11. Fixed bug to ensure the server still works on Java 5.
 

Change Log - Version 7.2.3

  1. Fixed bug preventing secret key authentication in some configurations.
  2. Made more handle tools aware of siteinfo.json file.
 

Change Log - Version 7.2.2

  1. Fixed vanishing rare deadlock in UDP message processing.
  2. Minor tweaks to resolver algorithm for picking one site among many in a service.
  3. Fixed packaging to allow easier building from source.
 

Change Log - Version 7.2.1

  1. Fixed issue with decryption of keys when starting newly-created servers.
  2. Fixed issue where certain handle values which should be cached would be resolved again every time.
  3. Ensured that hdl-setup-server script recreates siteinfo.bin if it existed previously (if not it may only create siteinfo.json).
 

Change Log - Version 7.2

IPv6:

  1. Handle servers can specify IPv6 addresses in their site information.
  2. For IPv6 address support, site info can use an attribute "alt_addr" to specify alternate addresses, allowing a single site info value to specify both an IPv4 and an IPv6 address for a server. In the Handle System, IPv6-capable sites should either use the "alt_addr" mechanism or the handle value type HS_SITE.6 so that the IPv6 addresses are not made visible to old handle clients which can not handle them.
  3. Implemented 'Happy Eyeballs' for IPv6 resolution.
  4. Added support for dual-stack servers in hdl-setup-server.

Server changes:

  1. Can now use editable siteinfo.json if siteinfo.bin is not present; create siteinfo.json during server setup.
  2. Made prefixes case-insensitive in authentication.
  3. Handle server now won't use JDBCache, instead uses MemCache.
  4. Improved indentation for config.dct.
  5. Handle servers now default to not performing recursive queries unless the "allow_recursion"="yes" setting is present.

Client tools changes:

  1. Allow config.dct in $HOME/.handle with "tcp_timeout", "no_udp_resolution", and "trace_resolution". (These keys match those from the handle server.)
  2. Added * to local_nas to use resolver_site for all requests, period.
  3. Enabled stopping handle server by deleting file delete_this_to_stop_server.
  4. Added hdl-dbresolve to resolve handles from local database.
  5. Fixed hdl-dbremove to work with new storage and to remove multiple handles.
  6. Added hdl-convert-siteinfo to convert siteinfo.bin to and from json for editing.
  7. fixed handle tool save console to add newlines.

API changes:

  1. Sessions are used for HSAdapter.
  2. Sessions are used for resolution requests with publicOnly = false.
  3. Versioning: set protocol version sensibly on requests sent directly to servers (rather than to a site).
  4. Client: Don't put session into session tracker until fully authenticated.
  5. Resolver will retry requests with new session when it receives session-timeout.

Protocol changes:

  1. New protocol flag overwriteWhenExists can be set on CreateHandleRequest to cause the values sent with the creation request to completely replace the value of an existing handle, similarly to deleting and re-creating; this flag can also be set on AddValueRequest to add new values and modify existing values in a single request.
  2. Use CBC instead of EBC for encryption. Protocol version 2.4. SessionInfos know their protocol version and decrypt and encrypt accordingly. File encryption (of private keys) can now use CBC as well.
  3. Change response codes of RC_SERVER_BACKUP and RC_NA_DELEGATE to match RFC, add compatibility for old versions (part of protocol version 2.5).
  4. Make messages from primary server have authoritative bit set (as per RFC).
  5. Warning on server when a client tries to authenticate twice in a session.
  6. Version 2.5 protocol. Defense against replay attacks: protocol version, session id, and req id are in signature, as well as a new "session counter" value; server ensures session counters are unique within a session. (Could have used request id but this allows request ids to be non-consecutive.)
  7. Reserved RC_ERROR for "something went wrong on the server". Use other response codes for "bad client message", notably RC_INVALID_VALUE which was in the RFC but not implemented.
  8. Added toErrorResponse() methods to HandleException to allow a potentially more sensible error response from recursive resolution.

Handle Replication Bug Fixes:

  1. Conserve unknown values in txnstat.dct.
  2. Change replicationDb to allow re-running a transaction.
  3. Fixed order of site timings so dump from fastest, not slowest.
  4. Fixed handle dumping to avoid potential loss of transaction in unusual circumstances.
  5. When redump is needed, don't change query date and start processing all transactions.
  6. Effort to fix non-streaming scanners on MySQL connectors.

Various Other Bug Fixes:

  1. Changed default cache TTL to 1 hour.
  2. Don't cache Values Not Found.
  3. Removed server references from files in client lib.
  4. Added some steps for Locale-independent case folding.
  5. Seed SecureRandom instances with System.nanoTime to avoid long wait for entropy.
  6. Fixed xml parsing quoting issues.
  7. Fixed http parameter parsing in local handle server http interface.
  8. Bug Fix to allow template handles with independent handle services.
 

Change Log - Version 7.1

  1. Ensure existing custom storage modules (such as DSpace) continue to function.
  2. Fixes made to replication and redumping from handle primary to secondary.
  3. Better error messages for generic batch without a passphrase.
  4. Handle servers now default to not performing recursive queries unless the allow_recursion=yes setting is present.
  5. SiteInfoConverter and hdl-convert-siteinfo to convert siteinfo.bin to and from JSON for command-line editing.
  6. Allow removing multiple handles with hdl-dbremove.
  7. Fix HandleResolver.sendRequestToServer (used by session setup and by replication) so that it tries all available protocols before throwing an exception.
  8. New protocol flag overwriteWhenExists, allowing Create requests to replace an existing handle.
  9. Bugfix to re-enable independent handle service.
  10. Turn off (by default) more debugging output.
 

Change Log - Version 7.0

  1. Created additional layer of verification of handle values by adding signature handle values that are signed with "offline" private keys (private keys that need not exist on any Internet-connected machine). These signatures are added to a handle's values and are used to sign other values in the same handle record. The signatures can be generated by third parties and handle clients using the SecureResolver class and can specify whose signatures are to be trusted. To begin with, the root service handle (0.NA/0.NA) and all prefix handle records will be signed, but users can choose to add signatures to their own individual handle records.
  2. Added "template" handles: handle records that don't actually exist in the database but are dynamically generated based on a stub handle. The dynamic generation of handle values is done by transforming the user-supplied extension to the "base" handle and constructing new values based on variables in the template or the requested handle.
  3. Added client support for new handle value type 10320/loc (loc being short for location) which specifies a list of (XML) attributed locations (URLs) to which a handle can resolve. The ability to select from among the multiple locations has been implemented in the proxy and Firefox extension, which now observe query parameters and other request parameters (such as client IP address, browser language, etc.) which affect selection of a specific URL.
  4. Added a DNS interface so that handle servers can be used to host DNS names. This includes support for translating DNS names to handles, using Punycode for decoding non-ASCII domain names.
  5. Included Berkeley DB JE jar library and made Berkeley DB JE the default database for new handle servers (i.e., servers without an existing handles.jdb file).
  6. Improved distribution layout including providing launch scripts in the bin directory (bin/doc/lib/etc). Using this new layout enables the creation of easy to use launch scripts, and the ability to add third party jar files and to separate specific functions into different jar files (i.e., client library, server code, Handle Tool).
  7. Numerous updates to enable a clean distribution build, including removing old security software that was bound to specific third party libraries.
  8. Added a simple but flexible API for building handle value digests and signing those digests (or other values, more generally).
  9. No longer assuming monotonically increasing request IDs in authenticated sessions. Request IDs are used by the client to determine which requests have been answered. Randomized request IDs prevent flooding UDP ports with fake responses that assume messages have incrementing request IDs.
  10. Made AuthenticationUtil able to authenticate without being given an explicit admin handle (key) index, which was required in prior versions. If the index is <= 0, it will try to verify the authentication using every public or secret key in the given handle record instead of just one.
  11. Added method to HandleStorageFactory to allow creation of read-only storage instances for storage modules that support a read-only mode (currently SQL and BDBJE).
  12. Added a configuration option for making BDBJE (Berkeley DB) read-only, allowing utilities to access the database while the server is running.
  13. Made changes for creating handles immediately upon creating a NA: (1) Capitalize 0.NA handles in the cache; (2) When looking up service info, give one extra effort to find the prefix authoritatively; (3) Look up (and thus cache) more admin-related value types when looking up service info. Also, fixed a bug in which looking up an NA for a handle did not use certified resolution.
  14. Made caching improvements: (1) Cache returns VALUE_NOT_FOUND response code instead of an empty SUCCESS response. (2) Cache caches VALUES_NOT_FOUND. (3) Cache will cache administrative reads, but only of the public values. (This last prevents an annoying occurrence in the Handle Tool, where you see new values when you edit a handle, but still see old values when you resolve it without editing.)
  15. Configuration option "log_save_directory" is now considered relative to the server directory, not the working directory, which prevents server log files from being generated in unexpected locations. Setup default is now to store logs in "logs".
  16. In proxy servlet, fixed Location (10320/loc) redirect handler so that it sees GET params, which were incorrectly ignored when determining a redirection.
  17. Corrected SQLHandleStorage, erroneously returning HANDLE_NOT_FOUND when it meant VALUES_NOT_FOUND.
  18. Small changes to HTMLFile and template system for proxy servlet.
  19. AuthenticationInfo.equals() now compares the key (to avoid possible hijacking issues; if an application compared AuthenticationInfo instances, a malicious user might have been able to use the same user identification but provide an incorrect key).
  20. Added "create handle" tester.
  21. Improved toString method of Transactions.
  22. Handle server stops sending response messages if the socket/connection has been broken rather than filling up the logs with IO errors while listing handles.
  23. Proxy: Set response code for robots.txt requests to SC_OK instead of SC_NOTFOUND .
  24. SessionSetupRequest no longer needs to be an admin request (can set up over UDP, and can set up with non-primary servers).
  25. Updated proxy code with "extra" log file and support for multiple actions when resolving, including "showurls", "showvalues" and "redirect".
  26. Trimmed spaces from the beginning or end of handle value types in Handle Tool.
  27. Removed overzealous HTML escaping in the proxy server.
  28. Treat 0.TYPE/URL and URL values as the same in the proxy server.
  29. Made handle data that looks like a URI clickable in the handle proxy's HTML display.
  30. HandleResolver now automatically removes cached entries for handles which are accessed by an administrator for editing, so that resolving those handles will retrieve the possibly changed values, rather than the cached values.
  31. Fixed problem with Handle Tool caching "not found" before it creates a handle.
  32. Made HandleResolver.setUdpRetryScheme public.
  33. Added Util.getPrivateKeyFromFileWithPassphrase.
  34. A resolution request which ignores restricted values will now return a VALUES_NOT_FOUND response if the request returns no values, but would return restricted values. Before this change it returned an empty SUCCESS.
  35. Added get/setTcpTimeout to net.handle.api classes.
  36. Pass a flag to the handle server java invocation to increase the upper limit for heap size and also use the -server hotspot configuration.
  37. Added Jython interpreter that can run script files or show an interactive shell. Command-line parameters can specify authentication information or prompt the user with a GUI authentication window.
  38. Added NamespaceInfo for tracking information related to the namespace (i.e., prefix or delegated subprefix) that contains any given handle. The HandleResolver will keep a reference to a chain of NamespaceInfo objects (most specific namespace first) when performing resolution.
  39. Location type handler now supports urlappend parameter.
  40. Change in build.xml file to make sure all net.handle.jdb.* classes are compiled with the base system.
  41. Fixed encryption encoding when a null secret key is given.
  42. Properly set hash code in takeValuesFrom method so that hash for challenge/response messages is encoded properly.
  43. Set isAdminRequest flag on home/unhome messages in batch file runner.
  44. Made sure the output file/stream is closed after listing all handles.
  45. Added query/admin status of interfaces to ServerInfo's toString() method.
  46. Proxy now returns hard-coded settings for robots.txt.
  47. Make proxy not convert + symbols in proxy URIs into spaces - (By request).
  48. Changed catch statement so that HDLProxy doesn't stop working if GeoIP library is not available.
  49. Added simple binary-to-hex command line app.
  50. Updated library (cnriutil.jar) to include leaner xml parser.
  51. Modified List Handles Request so that it is no longer an administrative request. This will make it so that clients will be able to do list handles on secondaries running newer versions of the server.
  52. Servlet proxy: made referrer logging the default.
  53. Integrated shadow password functionality into the admin tool. Users will now be able to authenticate using shadow passwords and create shadow password HS_SECKEY values. This stores a hash of the password in the handle record, rather than the password itself.
  54. Updated secret key value editor to disable public-read permission on HS_SECKEY values by default. Also clears the password field when changing the value.
  55. Made Handle Tool quit when closing the last window.
  56. Added functions to Handle Tool:
  57. Minor improvements to caching to remove some of the oddities seen with the Handle Tool. Now it ensures that the non-certified cache contains everything in the certified cache.
  58. Fixed problems with clearing caches in the Handle Tool and HandleResolver API.
  59. Made small improvements to handle listing tool.
  60. Fixed encryption of handle messages which prevented servers from being able to decrypt certain client messages properly.

Minor Bug Fixes:

July 24, 2015