Multiprocessing & Multithreading in Python

Intro

multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads. Due to this, the multiprocessing module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows.

The multiprocessing module provides a number of classes and functions that allow you to create and manage multiple processes in Python. Some of the key components of the multiprocessing module include:

• The Process class: a new process that can be started and run independently of the main process.
• The Pool class: provides a simple way to create a pool of worker processes that can be used to parallelize tasks across multiple CPU cores.
• The Queue class: provides a thread-safe way to pass messages and data between processes.
• The Lock class: provides a way to synchronize access to shared resources across multiple processes.

Handling accessing the same resource

One important thing to keep in mind when writing parallel code with multiprocessing is that you need to be careful about how you handle shared resources such as variables, files, and network connections. Since each process runs independently, it's possible…