fix(setup_helpers): ensure ThreadPool is closed (#3548)

* Ensure ThreadPool is closed in setup_helpers

The ParallelCompile setup helper using a ThreadPool to enable its
parallelism. It does not properly close the pool when it is done with
it.
This can lead to a "Exception ignored in: <function Pool.__del__..."
message with traceback being printed at shutdown.
Use pool.terminate() instead of context manager for Python 2.7
compatibility

* Add note to remove code that supports Python 2

Co-authored-by: Bobby Impollonia <bobby@k13capital.com>
This commit is contained in:
Bobby Impollonia 2021-12-13 12:45:33 -05:00 committed by GitHub
parent 59aa99860c
commit 7516811315
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -466,8 +466,14 @@ class ParallelCompile(object):
threads = 1 threads = 1
if threads > 1: if threads > 1:
for _ in ThreadPool(threads).imap_unordered(_single_compile, objects): pool = ThreadPool(threads)
# In Python 2, ThreadPool can't be used as a context manager.
# Once we are no longer supporting it, this can be 'with pool:'
try:
for _ in pool.imap_unordered(_single_compile, objects):
pass pass
finally:
pool.terminate()
else: else:
for ob in objects: for ob in objects:
_single_compile(ob) _single_compile(ob)