*** 24.0.19 bugs

- Fixed a bug where an Nvidia GPU that has been subdivided in MIG subdevices
  would not properly pass the device names in the various nvidia related
  environment variables.
  :jira:`3567`

- Fixed a bug for **batch** grid universe jobs where submitting to a remote
  system would fail if the username on the remote system contained a dot.
  :jira:`3581`

- Fixed a bug where some of the values reported for job execution time were
  very large if the Access Point daemons restarted while the job was running.
  :jira:`3592`

- The :meth:`htcondor2.send_command` method no longer incorrectly requires
  a `target` parameter.
  :jira:`3597`

*** 24.0.18 features

- Enable the in-memory cache option in newer versions of the scitokens-cpp
  library. This allows the library to continue operating when the on-disk
  cache is not useable.
  :jira:`3560`

- HTCondor tarballs now contain `Pelican 7.23.0 <https://pelicanplatform.org/releases>`_

- The condor package now requires pelican-7.23.0.

*** 24.0.18 bugs

- Fixed a bug in :tool:`condor_submit` that could expand the :subcom:`executable`
  path incorrectly when using late materialization and a different executable
  for each job.
  :jira:`3546`

- Fixed a bug where new STARTD_CRON values didn't trigger an immediate
  update of slot ads to the collector when configured to do so.
  :jira:`3526`

- Update Annex's knowledge about the Delta HPC system.
  :jira:`3559`

*** 24.0.17 bugs

- Fixed :meth:`htcondor2.Schedd.refreshGSIProxy` to use correct
  variables and not immediately fail.
  :jira:`3503`

- Fixed bugs in :tool:`condor_history` that made it very slow on
  filesystems with rate-limits on I/O operations.
  :jira:`3512`

*** 24.0.16 features

- HTCondor tarballs now contain `Pelican 7.22.0 <https://pelicanplatform.org/releases>`_

- The condor package now requires pelican-7.22.0.

- HTCondor tarballs now contain `Apptainer 1.4.5 <https://github.com/apptainer/apptainer/releases/tag/v1.4.5>`_

- The condor RPM package now requires at least apptainer version 1.4.5.

*** 24.0.16 bugs

- Fixed a bug where a job with a floating point value for :ad-attr:`RequestMemory`
  or :ad-attr:`RequestDisk` might not match the slot created to run it.
  :jira:`3423`

- Fixed a bug that would cause a crash in the *condor_rooster* and *condor_defrag* when many slots
  would have identical :macro:`ROOSTER_UNHIBERNATE_RANK` expressions, or if was
  not constant (e.g. used random() or time()).
  :jira:`3436`

- Removed memory leak from :meth:`htcondor2.JobEventLog.events`.
  :jira:`3474`

- Changed :tool:`condor_history` so that when it prints jobs in ``-long`` format
  it prints them with attributes sorted alphabetically like :tool:`condor_q` does.
  :jira:`3481`

- Fix Logical Volume setup timeout on Execution Points enforcing disk usage
  for jobs that have a large :subcom:`request_disk` value (``800+ GB``).
  :jira:`3432`

- Fixed issue with Execution Points enforcing disk usage generating an
  excessive amount of metadata archives filling up ``/etc/lvm/archive``.
  :jira:`3488`

- Added :macro:`FILETRANSFER_PLUGIN_CLASSAD_TIMEOUT` for administrators
  to set a longer timeout for plugins to detect their health.
  :jira:`3455`

- Fixed a bug where docker universe jobs would bring back a
  file named ``.docker_stderror`` back to the AP after job completion.
  :jira:`3424`

- Fixed a bug where the values of the attributes :ad-attr:`LocalJobsIdle` and
  :ad-attr:`LocalJobsRunning` in the submitter ad were reversed.
  :jira:`3456`

*** 24.0.15 features

- HTCondor tarballs now contain `Pelican 7.21.1 <https://pelicanplatform.org/releases>`_

- The condor package now requires pelican-7.21.1.

- HTCondor tarballs now contain `Apptainer 1.4.4 <https://github.com/apptainer/apptainer/releases/tag/v1.4.4>`_

- The condor RPM package now requires at least apptainer version 1.4.4.

*** 24.0.15 bugs

- Fixed a bug that could cause the AP to fail to read job credential files.
  :jira:`3377`

- Fixed bugs that could cause a crash in the authentication code.
  :jira:`3394`

- Fixed a bug where the *condor_negotiator* would fail to contact the
  *condor_schedd* to perform matchmaking if its :macro:`ALLOW_CLIENT`
  configuration parameter didn't authorize the identity
  ``submit-side@matchsession``.
  :jira:`3378`

- Fixed a bug where tools like :tool:`condor_status` would print an incorrect
  value for the ``-af:r`` option when the value to be printed was a single
  attribute reference.
  :jira:`3347`

*** 24.0.14 bugs

- Fix problem running PyTorch jobs on multiple GPUs with
  newer versions of the CUDA library by providing long GPU IDs
  in the ``CUDA_VISIBLE_DEVICES`` environment variable
  :jira:`3350`

- Calling :meth:`htcondor2.Schedd.history` on an object whose corresponding
  daemon can't be contacted will no longer cause a segmentation fault.
  :jira:`3314`

- Executions points using disk enforcement failed when provided backing
  :macro:`LVM_VOLUME_GROUP_NAME` and :macro:`LVM_THINPOOL_NAME` contained
  hyphens.
  :jira:`3334`

- Add a timeout for all HTTP operations in the grid universe (affects
  **arc**, **ec2**, and **gce** grid types).
  :jira:`3300`

- Fixed a bug where administrative tools (e.g. :tool:`condor_drain`) can fail if
  authentication is not required for queries to the *condor_collector*.
  :jira:`3301`

- The *condor_startd* now correctly removes docker containers that have escaped from
  docker universe jobs launched by *condor_starter*\'s that have exited uncleanly.
  :jira:`3338`

- Annexes no longer download a default configuration tarball, making them
  more robust.  This required upgrading the default version of HTCondor run
  by annexes to 25.1.0, which can figure the details out on its own.  On
  Delta, annexes now give Delta additional time to start file-transfer plug-ins,
  so they should be more-reliably available there.
  :jira:`3220`

- :tool:`condor_adstash` no longer skips entire history files when it
  encounters a line that cannot be decoded to UTF-8.
  :jira:`3335`

- Fixed a bug in *blahpd* when ``/tmp`` is mounted with the ``noexec`` option.
  :jira:`3343`

*** 24.0.13 bugs

- Calling :meth:`htcondor2.Schedd.history` on an object whose corresponding
  daemon can't be contacted will no longer cause a segfault.
  :jira:`3314`

*** 24.0.13 features

- HTCondor tarballs now contain `Pelican 7.20.2 <https://pelicanplatform.org/releases>`_

- The condor package now requires pelican-7.20.2.

*** 24.0.13 bugs

- Fixed a bug where space and comma would be included in the list of
  separators for itemdata even if the itemdata had been supplied with
  the ASCII "unit separator".  This would cause itemdata entries containing
  spaces (or commas) to be incorrectly interpreted as multiple items, which
  could manifest as parse errors.  You can work around this bug if only one
  of your entries has spaces and/or commas by moving that entry to the end
  of the line (or dictionary, if you're submitting itemdata via Python).
  :jira:`3272`

- Fixed false positive reporting of ClassAd unit specifier test in
  :tool:`condor_upgrade_check`.
  :jira:`3276`

*** 24.0.12 features

- Updated :tool:`condor_upgrade_check` to test for well known gotchas
  between v24 and v25 of HTCondor installations.
  :jira:`3209`

*** 24.0.12 bugs

- Fixed a bug where the *blahpd* would fail when using Python 3.12 or later.
  :jira:`3225`

- Fixed bug where jobs held due to exceeding disk usage had the same
  :ad-attr:`HoldReasonCode` and :ad-attr:`HoldReasonSubCode` as jobs
  that exceeded memory usage.
  :jira:`3248`

*** 24.0.11 features

- Initial support for Debian 13 (trixie).
  :jira:`3212`

*** 24.0.11 bugs

*** 24.0.10 bugs

*** 24.0.9 features

- Initial Support for Enterprise Linux 10, including the x86_64_v2 platform.
  :jira:`3090`

*** 24.0.9 bugs

- The results of ``key in htcondor2.param`` and
  ``key in htcondor2.param.keys()`` now match for keys which are defined to
  have no value.  (Previously, such keys would be returned by ``keys()``.)
  :jira:`3085`

*** 24.0.8 bugs

- Fixed a bug introduced in 24.0.7 and 24.7.3 when running on Linux cgroup v1
  systems, jobs that were killed by the out-of-memory killer were
  considered completed instead of being put on hold.
  :jira:`3094`

- The :class:`htcondor2.Credd` initializer now properly raised
  a :class:`TypeError` if the location argument isn't
  a :class:`classad2.ClassAd`, rather than failing to raise a ``TypError``.
  :jira:`2993`

*** 24.0.7 features

- The *condor_stard* now distributes the :ad-attr:`LoadAvg` assigned to a partitionable slot
  to the idle resources of the partitionable slot, and then to the dynamic slots.
  Machines that have only a single partitionable slot will now have the same behavior under
  a DESKTOP policy as they did in version 23.10.18 and 24.0.1.
  :jira:`2901`

*** 24.0.7 bugs

- When using delegated cgroup v2, HTCondor no longer reports that that main job (often a pilot)
  has an out of memory condition when only the sub-job has hit an oom.
  :jira:`2944`

*** 24.0.5 bugs

*** 24.0.4 features

- For **arc** grid universe jobs, the new submit command
  :subcom:`arc_data_staging` can be used to supply additional elements
  to the DataStaging block of the ARC ADL that HTCondor constructs.
  :jira:`2774`

*** 24.0.4 bugs

- Fixed a bug in the negotiator that caused it to crash when matching
  offline ads.
  :jira:`2819`

- Fixed a memory leak in the schedd that could be caused by SCHED_CRON
  scripts that generate standard error output.
  :jira:`2817`

- Fixed a bug that cause the *condor_schedd* to crash with a segmentation
  fault if a :tool:`condor_off` -fast command was run while a schedd cron
  script was running.
  :jira:`2815`
