["python","parallel-processing"]
How do I parallelize a simple Python loop?

This is probably a trivial question, but how do I parallelize the following loop in python? # setup output lists output1 = list() output2 = list() output3 = list() for j in range(0, 10): # calc individual parameter value parameter = j * offset # call the calculation out1, out2, out3 = calc_stuff(parameter = parameter) # put results into correct output list output1.append(out1) output2.append(out2) output3.append(out3) I know how to start single threads in Python but I don't know how to "collect" the results. Multiple processes would be fine too - whatever is easiest for this case. I'm using currently Linux but the code should run on Windows and Mac as-well. What's the easiest way to parallelize this code?
The CPython implementation currently has a global interpreter lock (GIL) that prevents threads of the same interpreter from concurrently executing Python code. This means CPython threads are useful for concurrent I/O-bound workloads, but usually not for CPU-bound workloads. The naming calc_stuff() indicates that your workload is CPU-bound, so you want to use multiple processes here (which is often the better solution for CPU-bound workloads anyway, regardless of the GIL). There are two easy ways of creating a process pool into the Python standard library. The first one is the multiprocessing module, which can be used like this: pool = multiprocessing.Pool(4) out1, out2, out3 = zip(*pool.map(calc_stuff, range(0, 10 * offset, offset))) Note that this won't work in the interactive interpreter due to the way multiprocessing is implemented. The second way to create a process pool is concurrent.futures.ProcessPoolExecutor: with concurrent.futures.ProcessPoolExecutor() as pool: out1, out2, out3 = zip(*pool.map(calc_stuff, range(0, 10 * offset, offset))) This uses the multiprocessing module under the hood, so it behaves identically to the first version.