Merge remote-tracking branch 'upstream/main' into fix-issue-138978

Author: skv0zsneg

Doc/library/ast.rst

@@ -2254,7 +2254,7 @@ and classes for traversing abstract syntax trees:
 
    In addition, if ``mode`` is ``'func_type'``, the input syntax is
    modified to correspond to :pep:`484` "signature type comments",
-   e.g. ``(str, int) -> List[str]``.
+   for example ``(str, int) -> List[str]``.
 
    Setting ``feature_version`` to a tuple ``(major, minor)`` will result in
    a "best-effort" attempt to parse using that Python version's grammar.

Doc/whatsnew/3.16.rst

@@ -120,6 +120,16 @@ codecs
   even when a built-in codec of the same name exists.
   (Contributed by Serhiy Storchaka in :gh:`152997`.)
 
+
+concurrent.futures
+------------------
+
+* The iterator returned by :meth:`concurrent.futures.Executor.map` is no longer
+  automatically closed if a function call raises an exception.
+  Use method :meth:`!close` to explicitly close the iterator.
+  (Contributed by xzmeng and Serhiy Storchaka in :gh:`108518`.)
+
+
 csv
 ---
 
@@ -140,6 +150,19 @@ csv
   instead of always returning ``'\r\n'``.
   (Contributed by Serhiy Storchaka in :gh:`75008`.)
 
+
+ctypes
+------
+
+* Add :func:`ctypes.util.struct` for generating :class:`~ctypes.Structure` types
+  from an annotation-based syntax, similar to how the :mod:`dataclasses` module
+  is used.
+  (Contributed by Peter Bierma in :gh:`104533`.)
+* Add :func:`ctypes.util.wrap_dll_function` for generating function pointers
+  through a function signature.
+  (Contributed by Peter Bierma in :gh:`153903`.)
+
+
 curses
 ------
 
@@ -274,27 +297,6 @@ curses
   (Contributed by Serhiy Storchaka in :gh:`133031`.)
 
 
-ctypes
-------
-
-* Add :func:`ctypes.util.struct` for generating :class:`~ctypes.Structure` types
-  from an annotation-based syntax, similar to how the :mod:`dataclasses` module
-  is used.
-  (Contributed by Peter Bierma in :gh:`104533`.)
-* Add :func:`ctypes.util.wrap_dll_function` for generating function pointers
-  through a function signature.
-  (Contributed by Peter Bierma in :gh:`153903`.)
-
-
-concurrent.futures
-------------------
-
-* The iterator returned by :meth:`concurrent.futures.Executor.map` is no longer
-  automatically closed if a function call raises an exception.
-  Use method :meth:`!close` to explicitly close the iterator.
-  (Contributed by xzmeng and Serhiy Storchaka in :gh:`108518`.)
-
-
 encodings
 ---------
 
@@ -310,12 +312,6 @@ gzip
   which is passed on to the constructor of the :class:`~gzip.GzipFile` class.
   (Contributed by Marin Misur in :gh:`91372`.)
 
-io
---
-
-* Add :meth:`io.BytesIO.peek` method to read without advancing position.
-  (Contributed by Marcel Martin in :gh:`90533`.)
-
 
 imaplib
 -------
@@ -368,12 +364,20 @@ imaplib
   (Contributed by Serhiy Storchaka in :gh:`153521`.)
 
 
+io
+--
+
+* Add :meth:`io.BytesIO.peek` method to read without advancing position.
+  (Contributed by Marcel Martin in :gh:`90533`.)
+
+
 ipaddress
 ---------
 
-Add :meth:`~ipaddress.IPv4Network.next_network` and
-:meth:`~ipaddress.IPv6Network.next_network` methods to find the next nearest
-network with a specific prefix size.
+* Add :meth:`~ipaddress.IPv4Network.next_network` and
+  :meth:`~ipaddress.IPv6Network.next_network` methods to find the next nearest
+  network with a specific prefix size.
+  (Contributed by Faisal Mahmood in :gh:`87027`.)
 
 
 logging
@@ -413,16 +417,15 @@ math
 multiprocessing
 ---------------
 
-* Add the optional ``buffersize`` parameter to
+* Add the optional *buffersize* parameter to
   :meth:`multiprocessing.pool.Pool.imap` and
   :meth:`multiprocessing.pool.Pool.imap_unordered` to limit the number of
   submitted tasks whose results have not yet been yielded. If the buffer is
-  full, iteration over the *iterables* pauses until a result is yielded from
+  full, iteration over the *iterable* pauses until a result is yielded from
   the buffer. To fully utilize pool's capacity when using this feature, set
   *buffersize* at least to the number of processes in pool (to consume
   *iterable* as you go), or even higher (to prefetch the next
   ``N=buffersize-processes`` arguments).
-
   (Contributed by Oleksandr Baltian in :gh:`136871`.)
 
 
@@ -471,6 +474,7 @@ shlex
   a string, even if it is already safe for a shell without being quoted.
   (Contributed by Jay Berry in :gh:`148846`.)
 
+
 sqlite3
 -------
 
@@ -479,6 +483,7 @@ sqlite3
   :exc:`SystemError` or :exc:`ValueError`.
   (Contributed by Jiseok CHOI in :gh:`150449`.)
 
+
 symtable
 --------
 
@@ -673,12 +678,6 @@ re
   (Contributed by Serhiy Storchaka in :gh:`152033` and Pieter Eendebak in
   :gh:`152056`.)
 
-module_name
------------
-
-* TODO
-
-
 
 Removed
 ========
@@ -719,22 +718,27 @@ asyncio
 functools
 ---------
 
-* Calling the Python implementation of :func:`functools.reduce` with *function*
-  or *sequence* as keyword arguments has been deprecated since Python 3.14.
+* Remove support for calling the Python implementation of
+  :func:`functools.reduce` with *function* or *sequence* as keyword arguments,
+  deprecated since Python 3.14.  These parameters are now positional-only.
+  (Contributed by Nikita Sobolev in :gh:`149537`.)
 
 logging
 -------
 
-* Support for custom logging handlers with the *strm* argument is deprecated
-  and scheduled for removal in Python 3.16. Define handlers with the *stream*
-  argument instead.
+* Remove support for custom logging handlers with the *strm* argument,
+  deprecated since Python 3.14.
+  Define handlers with the *stream* argument instead.
+  (Contributed by Nikita Sobolev in :gh:`149598`.)
 
 mimetypes
 ---------
 
-* Valid extensions start with a '.' or are empty for
-  :meth:`mimetypes.MimeTypes.add_type`.
-  Undotted extensions now raise a :exc:`ValueError`.
+* Remove support for undotted extensions in
+  :meth:`mimetypes.MimeTypes.add_type`, deprecated since Python 3.14.
+  Valid extensions must now start with a ``'.'`` or be empty;
+  undotted extensions raise a :exc:`ValueError`.
+  (Contributed by Nikita Sobolev in :gh:`149720`.)
 
 shutil
 ------
@@ -791,14 +795,14 @@ Deprecated
 New deprecations
 ----------------
 
-* :mod:`abc`
+* :mod:`abc`:
 
-  * Soft-deprecated since Python 3.3 :class:`abc.abstractclassmethod`,
-    :class:`abc.abstractstaticmethod`, and :class:`abc.abstractproperty`
+  * :class:`abc.abstractclassmethod`, :class:`abc.abstractstaticmethod`,
+    and :class:`abc.abstractproperty`, soft-deprecated since Python 3.3,
     now raise a :exc:`DeprecationWarning`.
-    These classes will be removed in Python 3.21, instead
+    These classes will be removed in Python 3.21;
     use :func:`abc.abstractmethod` with :func:`classmethod`,
-    :func:`staticmethod`, and :class:`property` respectively.
+    :func:`staticmethod`, and :class:`property` respectively instead.
 
 * :mod:`ast`:
 
@@ -813,12 +817,12 @@ New deprecations
 
 * :mod:`struct`:
 
-  * Soft-deprecated since Python 3.15, using ``'F'`` and ``'D'`` type codes are now
-    deprecated.  These codes will be removed in Python 3.21.  Use instead
-    two-letter forms ``'Zf'`` and ``'Zd'``.
+  * The ``'F'`` and ``'D'`` type codes, soft-deprecated since Python 3.15,
+    are now deprecated.  These codes will be removed in Python 3.21.
+    Use the two-letter forms ``'Zf'`` and ``'Zd'`` instead.
     (Contributed by Sergey B Kirpichev in :gh:`121249`.)
 
-* :mod:`tempfile`
+* :mod:`tempfile`:
 
   * The private ``tempfile._TemporaryFileWrapper`` name is deprecated
     and is slated for removal in Python 3.21.

Lib/asyncio/taskgroups.py

@@ -239,6 +239,9 @@ def create_task(self, coro, **kwargs):
         # the current task too early. gh-128550, gh-128588
         self._tasks.add(task)
         task.add_done_callback(self._on_task_done)
+        # gh-155418: an eager task can cancel the group before joining _tasks
+        if self._aborting and not task.done():
+            task.cancel()
         try:
             return task
         finally:

Lib/enum.py

@@ -624,9 +624,13 @@ def __new__(metacls, cls, bases, classdict, *, boundary=None, _simple=False, **k
                     '__invert__'
                 ):
                 if name not in classdict:
+                    # check for mixin overrides before replacing
                     enum_method = getattr(Flag, name)
-                    setattr(enum_class, name, enum_method)
-                    classdict[name] = enum_method
+                    found_method = getattr(enum_class, name)
+                    data_type_method = getattr(member_type, name, None)
+                    if found_method in (enum_method, data_type_method):
+                        setattr(enum_class, name, enum_method)
+                        classdict[name] = enum_method
         #
         # replace any other __new__ with our own (as long as Enum is not None,
         # anyway) -- again, this is to support pickle

Lib/http/cookiejar.py

@@ -627,7 +627,7 @@ def request_host(request):
 
     """
     url = request.get_full_url()
-    host = urllib.parse.urlparse(url)[1]
+    host = urllib.parse.urlparse(url).netloc
     if host == "":
         host = request.get_header("Host", "")
 

Lib/test/libregrtest/refleak.py

@@ -1,6 +1,7 @@
 import os
 import sys
 import warnings
+from array import array
 from inspect import isabstract
 from typing import Any
 import linecache
@@ -100,24 +101,20 @@ def runtest_refleak(test_name, test_func,
     for obj in ByteString.__subclasses__() + [ByteString]:  # type: ignore[attr-defined]
         abcs[obj] = _get_dump(obj)[0]
 
-    # bpo-31217: Integer pool to get a single integer object for the same
-    # value. The pool is used to prevent false alarm when checking for memory
-    # block leaks. Fill the pool with values in -1000..1000 which are the most
-    # common (reference, memory block, file descriptor) differences.
-    int_pool = {value: value for value in range(-1000, 1000)}
-    def get_pooled_int(value):
-        return int_pool.setdefault(value, value)
-
     warmups = hunt_refleak.warmups
     runs = hunt_refleak.runs
     filename = hunt_refleak.filename
     repcount = warmups + runs
 
-    # Pre-allocate to ensure that the loop doesn't allocate anything new
+    # Pre-allocate to ensure that the loop doesn't allocate anything new.
+    # Store the deltas as raw values in arrays rather than as int objects in
+    # lists: each unique delta stored as an object would live until the end of
+    # the loop and show up in the following repetition's reference and memory
+    # block deltas (gh-75400, gh-155981).
     rep_range = list(range(repcount))
-    rc_deltas = [0] * repcount
-    alloc_deltas = [0] * repcount
-    fd_deltas = [0] * repcount
+    rc_deltas = array('q', [0]) * repcount
+    alloc_deltas = array('q', [0]) * repcount
+    fd_deltas = array('q', [0]) * repcount
     getallocatedblocks = sys.getallocatedblocks
     gettotalrefcount = sys.gettotalrefcount
     getunicodeinternedsize = sys.getunicodeinternedsize
@@ -161,12 +158,11 @@ def get_pooled_int(value):
         rc_after = gettotalrefcount()
         fd_after = fd_count()
 
-        rc_deltas[i] = get_pooled_int(rc_after - rc_before)
-        alloc_deltas[i] = get_pooled_int(alloc_after - alloc_before)
-        fd_deltas[i] = get_pooled_int(fd_after - fd_before)
+        rc_deltas[i] = rc_after - rc_before
+        alloc_deltas[i] = alloc_after - alloc_before
+        fd_deltas[i] = fd_after - fd_before
 
         if not quiet:
-            # use max, not sum, so total_leaks is one of the pooled ints
             total_leaks = max(rc_deltas[i], alloc_deltas[i], fd_deltas[i])
             if total_leaks <= 0:
                 symbol = '.'
@@ -212,13 +208,13 @@ def check_fd_deltas(deltas):
         return any(deltas)
 
     failed = False
-    for deltas, item_name, checker in [
+    for raw_deltas, item_name, checker in [
         (rc_deltas, 'references', check_rc_deltas),
         (alloc_deltas, 'memory blocks', check_rc_deltas),
         (fd_deltas, 'file descriptors', check_fd_deltas)
     ]:
-        # ignore warmup runs
-        deltas = deltas[warmups:]
+        # ignore warmup runs; convert to a list for reporting
+        deltas = list(raw_deltas[warmups:])
         failing = checker(deltas)
         suspicious = any(deltas)
         if failing or suspicious:

Lib/test/ssl_servers.py

@@ -61,7 +61,7 @@ def translate_path(self, path):
 
         """
         # abandon query parameters
-        path = urllib.parse.urlparse(path)[2]
+        path = urllib.parse.urlparse(path).path
         path = os.path.normpath(urllib.parse.unquote(path))
         words = path.split('/')
         words = filter(None, words)

Lib/test/support/__init__.py

@@ -868,7 +868,7 @@ def open_urlresource(url, *args, **kw):
 
     check = kw.pop('check', None)
 
-    filename = urllib.parse.urlparse(url)[2].split('/')[-1] # '/': it's URL!
+    filename = urllib.parse.urlparse(url).path.split('/')[-1] # '/': it's URL!
 
     fn = os.path.join(TEST_DATA_DIR, filename)
 

Lib/test/support/os_helper.py

@@ -434,7 +434,10 @@ def _rmtree_inner(path):
                           file=sys.__stderr__)
                     mode = 0
                 if stat.S_ISDIR(mode):
-                    _waitfor(_rmtree_inner, fullname, waitall=True)
+                    # Do not follow junctions, which os.lstat() reports
+                    # as directories.
+                    if not os.path.isjunction(fullname):
+                        _waitfor(_rmtree_inner, fullname, waitall=True)
                     _force_run(fullname, os.rmdir, fullname)
                 else:
                     _force_run(fullname, os.unlink, fullname)

Lib/test/test_ast/test_ast.py

@@ -162,6 +162,15 @@ def test_parse_invalid_ast(self):
             self.assertRaises(TypeError, ast.parse, ast.Constant(42),
                               optimize=optval)
 
+    def test_parse_ast_func_type(self):
+        # see gh-156689
+        tree = ast.parse('(int, str) -> bool', mode='func_type')
+        self.assertEqual(ast.dump(ast.parse(tree, mode='func_type')),
+                         ast.dump(tree))
+        self.assertRaises(TypeError, ast.parse, ast.Constant(42),
+                          mode='func_type')
+        self.assertRaises(TypeError, ast.parse, tree, mode='exec')
+
     def test_optimization_levels__debug__(self):
         cases = [(-1, '__debug__'), (0, '__debug__'), (1, False), (2, False)]
         for (optval, expected) in cases: