# Kea 1.7.7, April 29th 2020, Release Notes Welcome to Kea 1.7.7, a monthly development release of Kea. This is the first public release that offers multithreading support. As with any other development release, use this with caution. Development releases are not recommended for production use. There are many changes introduced in this version: 1. **Multithreading**. Our work continues on MT support improvements. The temporary command-line option -N has now been replaced with more extensive configuration options defined in the configuration file: `enable-multi-threading`, `thread-pool-size`, and `packet-queue-size` #893. A problem that causes roughly 0.5% of DHCPv6 responses to not be returned has been fixed #1158. The default logging now includes thread-id #1150. A number of problems reported by thread sanitizer have been fixed in the Control Agent #1094, libprocess #1096, and libutil #1089. The socket control buffer is now kept on the stack #343. Finally, a set of performance tests results has been [published](https://kb.isc.org/docs/kea-performance-tests-17-multithreadi ng). This may be of use for people interested in MT and performance, so it's mentioned here. 2. **Prometheus exporter and Grafana template**. A new project is under development: ISC Stork is a dashboard that provides Kea monitoring and troubleshooting. It includes a Stork agent that provides Prometheus exporter capability. You can run this agent to periodically retrieve statistics from Kea and make them available in a format that Prometheus can understand. Stork also includes a Grafana template that can be easily imported into Grafana to visualize the data exported to Prometheus. While technically Stork is a stand-alone project, it may be perceived by many as a Kea extension. As such, it is mentioned in these release notes #279, #280, #1191. 3. **Performance improvements**. Our team was able to address two small performance improvements this month, in addition to all the MT work mentioned earlier. MapElement is an internal representation of JSON and is used in many places in the code; we improved how its instances are compared #1187. Another improvement is related to statistics; under certain circumstances, statistics were calculated multiple times. This is now fixed #1162. 4. **Forensic logging**. The library can now print values as plain text if the content is printable #949. 5. **Package improvements**. The RADIUS package now depends on host cache #1157. A problem with Control Agent sometimes failing to start was traced to a race condition between systemd scripts. The improved dependency fixed the problem #1134. 6. **New Statistics**. Several new statistics have been implemented. `cumulative-assigned-addresses` provides an ever-growing number of leases that have been assigned. This can be used e.g. to monitor the number of devices visiting a network daily. Similar statistics have also been added for IPv6 addresses (`cumulative-assigned-nas`) and IPv6 prefixes (`cumulative-assigned-pds`). The statistics are available globally and on a per-subnet basis #816. 7. **Leasequery**. The work on leasequery support is picking up. The design is now complete #994 and the code is able to store more lease parameters with the lease #1010. In the coming weeks, we will implement the code to actually use that ability to return more detailed information in leasequery responses. 8. **Notable bug fixes**. An unlikely MySQL InnoDB deadlock has been fixed. While the possibility had existed for a long time, the deadlock was unlikely to be triggered due to the single-threaded nature of earlier Kea versions. With multithreading now available the likelihood of triggering this problem was elevated, but the problem is now fixed #1160. A problem with the RADIUS hook failing to start has been fixed #1156. The Flex-id hook now has better sanity checks for misconfigured expressions; previously, the hook loaded but then reported errors for every incoming packet if the expression configured was incorrect #1083. Another sporadically occurring problem with Kea-dhcp4 sometimes freezing on start has been fixed. This one is related to how Linux handles LPF sockets; if Kea opened a socket too early on an interface that wasn't ready to receive traffic, the read operation on LPF socket could get stuck in infinite wait #1151. 9. **Documentation** The multithreading section of the Kea ARM now mentions MT parameters and its defaults #1192, and a section with pointers to Stork has been added #1191. `lease4-get-page` and `lease6-get-page` commands that were implemented some time ago now have their documentation generated properly #882. 10. **Testing improvements**. Perfdhcp is now able to simulate traffic coming in from multiple subnets. A new command-line option `-J` allows passing a filename that contains IPv4 addresses of DHCP relays to be simulated. Kea-dhcp4 can be forced to send back responses to the source address of the incoming reply. While technically this violates the RFC standard, it is very useful for testing #1132. ## Changes to Release Model The Kea project has been in development for several years now, and it has a significant production deployment base with users who are looking for stability, rather than a constant stream of new "bleeding-edge" features. At the same time, we want to continue developing the software and add some new powerful, but difficult-to-implement, features. As a result, we decided to change the release cycle. Starting from 1.6.0, there are two branches: stable and development. Stable releases are what you would expect: stable, released infrequently, without new features or significant changes, very well-tested. These can be identified by an even-numbered major version number. The current stable release is 1.6.2, with a major version of 1.6, and minor version 2. If we discover important bugs that require fixing, we may release 1.6.3, but that will be determined on a case-by-case basis. The next major stable version will be 1.8.0, followed by 2.0.0 in the future. Our team continues development of new features. In particular, we're tackling the difficult problem of being able to use all available CPU cores simultaneously. The multithreading implementation is a complex task and it is unknown how long it will take before the solution is stable and ready for a production environment. At the same time, we continue to receive a stream of requests for small features and bug fixes. We don't want to force users to wait half a year or more for the fixes and features that are already done. Therefore, we have started issuing development releases on a monthly basis. Those are slightly less well-tested and may have features that are not complete. Development releases can be easily identified by an odd major version number: for example, 1.7.7 is a development release. In June 2020 we expect to release 1.7.8, the next development version. Once 1.8.0 is out, we will continue our development work with 1.9.0, then 1.9.1, and so on. Our goal is to make the development release available on the last Wednesday of each month. There may be exceptions (such as during holidays), but that's the general plan. We encourage users to test the development releases and report back their findings. For more details on the plan, see ISC's Software Support Policy at https://kb.isc.org/docs/aa-00896. ## Kea overview Kea is a DHCP implementation developed by Internet Systems Consortium, Inc. that features fully functional DHCPv4 and DHCPv6 servers, a dynamic DNS update daemon, a Control Agent (CA) that provides a REST API to control the DHCP and DNS update servers, an example shell client to connect to the CA, a daemon that is able to retrieve YANG configuration and updates from Sysrepo, and a DHCP performance-measurement tool. Both DHCP servers fully support server discovery, address assignment, renewal, rebinding, release, decline, information request, DNS updates, client classification, and host reservations. The DHCPv6 server also supports prefix delegation. Lease information is stored in a CSV file by default; it can optionally be stored in a MySQL, PostgreSQL, or Cassandra database instead. Host reservations can be stored in a configuration file, or in a MySQL, PostgreSQL, or Cassandra database. They can also be retrieved from a RADIUS server, although this functionality is somewhat limited. Kea DHCPv4 and DHCPv6 daemons provide support for YANG models, which are stored in a Sysrepo datastore and can be configured via the NETCONF protocol. This text references issue numbers. For more details, visit the Kea GitLab page at https://gitlab.isc.org/isc-projects/kea/issues. ## License This version of Kea is released under the Mozilla Public License, version 2.0. https://www.mozilla.org/en-US/MPL/2.0 The premium and subscriber-only hook libraries are provided in source code form, under the terms of an End User License Agreement (you will get the source code that you can modify freely, but you are not permitted to redistribute it). ## Download Pre-built ISC packages for current versions of the most popular Linux operating systems are available at: https://cloudsmith.io/~isc/repos/ The Kea source and PGP signature for this release may be downloaded from: https://www.isc.org/download The signature was generated with the ISC code signing key which is available at: https://www.isc.org/pgpkey ISC provides detailed documentation, including installation instructions and usage tutorials, in the Kea Administrator Reference Manual. Documentation is included with the installation, at https://kea.readthedocs.io/en/latest/, or via https://kb.isc.org/docs/kea-administrator-reference-manual in HTML, plain text, or PDF formats. ISC maintains a public open source code tree, a wiki, an issue tracking system, milestone planning, and a roadmap at https://gitlab.isc.org/isc-projects/kea. Limitations and known issues with this release can be found at https://gitlab.isc.org/isc-projects/kea/wikis/known-issues-list. We ask users of this software to please let us know how it worked for you and what operating system you tested on. Feel free to share your feedback on the Kea Users mailing list (https://lists.isc.org/mailman/listinfo/kea-users). Also we would like to hear whether the documentation is adequate and accurate. Please open tickets in the Kea GitLab project for bugs, documentation omissions and errors, and enhancement requests. We want to hear from you even if everything worked. ## Support Professional support for Kea is available from ISC. We encourage all professional users to consider this option; Kea development and maintenance are funded with support subscriptions. For more information on ISC's Kea and DHCP software support see https://www.isc.org/support/. Free best-effort support is provided by our user community via a mailing list. Information on all public email lists is available at https://www.isc.org/community/mailing-list. If you have any comments or questions about working with Kea, please share them to the Kea Users List (https://lists.isc.org/mailman/listinfo/kea-users). Bugs and feature requests may be submitted via GitLab at https://gitlab.isc.org/isc-projects/kea/issues. ## Changes The following summarizes changes and important upgrade notes since the previous release (1.7.6). 1743. [doc] tomek ISC now provides a dashboard solution for Kea. Added documentation pointers about the ISC Stork project. (Gitlab #1191) 1742. [func] fdupont Added new cumulative statistics: for each assigned IPv4 address, IPv6 NA address or IPv6 PD prefixes cumulative-assigned-addresses, cumulative-assigned-nas or cumulative-assigned-pds global and per subnet is increased. A spelling error in the result-set of the stat-lease4-get of the stat cmds hook library was fixed. (Gitlab #816) 1741. [func] fdupont When the MySQL InnoDB reports a deadlock (very rare event) the failed query is rerun. (Gitlab #1160) 1740. [bug] fdupont Incorrect signal handling in the process library (used for control agent, d2 and netconf server but not by DHCPv4 or DHCPv6 servers) was replaced by the boost ASIO signal set. (Gitlab #1096) 1739. [func] razvan Added multi threading configuration parameters which can be used to adjust server performance according to setup resources and needs: enable-multi-threading, thread-pool-size and packet-queue-size. There is also an automatic detection implemented by using thread-pool-size set to 0. However, the packet-queue-size must be manually configured and does not scale automatically. The temporary -N command line option was removed. The multi threading is disabled by default. (Gitlab #893) 1738. [func] fdupont Added new global parameters statistic-default-sample-count and statistic-default-sample-age to set the default values used for sample keeping limits. (Gitlab #1005) 1737. [func] tmark Added store-extended-info parameter to kea-dhcp4 and kea-dhcp6 which enables the storage of additional information per lease. This has been added in anticipation of supporting LeaseQuery. (Gitlab #1010) 1736. [doc] fdupont A default logging syntax has been updated to reflect recently added multi-threading capabilities. The process is now identified by name/process-id.thread-id, e.g. kea-dhcp4.dhcpsrv/8475.12345, where 12345 is the thread-id. Its exact syntax is OS dependent. The logging syntax can be tweaked using `pattern` option for logging. (Gitlab #1150) Thank you again to everyone who assisted us in making this release possible. We look forward to receiving your feedback.