Concurrency and Asynchronous Programming

This section describes what is meant by Concurrency and Asynchronous Programming. The presentation by Robert Smallshire provides a nice overview of concurrent programming and Python’s asyncio module.

Concurrency

Concurrent programming uses a single thread to execute multiple tasks in an interleaved fashion. This is different from parallel programming where multiple tasks can be executed at the same time.

_images/concurrency_vs_parallelism.png

The Network Manager uses concurrent programming. It runs in a single event loop but it can handle multiple Clients and Services connected to it simultaneously.

When a Client sends a request, the Manager forwards the request to the appropriate Service and then the Manager waits for another event to occur. Whether the event is a reply from a Service, another request from a Client or a new device wanting to connect to the Manager, the Manager simply waits for I/O events and forwards an event to the appropriate network device when an event becomes available.

Since the Manager is running in a single thread it can only process one event at a single instance in time. In typical use cases, this does not inhibit the performance of the Manager since the Manager has the sole responsibility of routing requests and replies through the network and it does not actually execute a request. There are rare situations when an administrator is making a request for the Manager to execute and in these situations the Manager would be executing the request, see admin_request() for more details.

The Manager can become slow if it is (de)serializing a large JSON object or sending a large amount of bytes through the network. For example, if a reply from a Service is 1 GB in size and the network speed is 1 Gbps (125 MB/s) then it will take at least 8 seconds for the data to be transmitted. During these 8 seconds the Manager will be unresponsive to other events until it finishes sending all 1 GB of data.

If the request for, or reply from, a Service consumes a lot of the processing time of the Manager it is best to start another instance of the Manager on another port to host the Service.

Asynchronous Programming

A Client can send requests either synchronously or asynchronously. Synchronous requests are sent sequentially and the Client must wait to receive the reply before proceeding to send the next request. These are blocking requests where the total execution time to receive all replies is the combined sum of executing each request individually. Asynchronous requests do not wait for the reply but immediately return a Future instance, which is an object that is a promise that a result (or exception) will be available later. These are non-blocking requests where the total execution time to receive all replies is equal to the time it takes to execute the longest-running request.

_images/sync_vs_async.png

Synchronous Example

The following code illustrates how to send requests synchronously. Before you can run this example on your own computer make sure to Start the Network Manager and start the BasicMath Service.

# synchronous.py
#
# This script takes about 21 seconds to run.

import time
from msl.network import connect

# Connect to the Manager (that is running on the same computer)
cxn = connect()

# Establish a link to the BasicMath Service
bm = cxn.link('BasicMath')

# Get the start time before sending the requests
t0 = time.perf_counter()

# Send all requests synchronously
# The returned object is the result of each request
add = bm.add(1, 2)
subtract = bm.subtract(1, 2)
multiply = bm.multiply(1, 2)
divide = bm.divide(1, 2)
is_positive = bm.ensure_positive(1)
power = bm.power(2, 4)

# Print the results
print(f'1+2= {add}')
print(f'1-2= {subtract}')
print(f'1*2= {multiply}')
print(f'1/2= {divide}')
print(f'is positive? {is_positive}')
print(f'2**4= {power}')

# The total time that passed to receive all results
dt = time.perf_counter() - t0
print(f'Total execution time: {dt:.2f} seconds')

# Disconnect from the Manager
cxn.disconnect()

The output of the synchronous.py program will be:

1+2= 3
1-2= -1
1*2= 2
1/2= 0.5
is positive? True
2**4= 16
Total execution time: 21.06 seconds

The Total execution time value will be slightly different for you, but the important thing to notice is that executing all requests took about 21 seconds (i.e., 1+2+3+4+5+6=21 for the time.sleep() functions in the BasicMath Service) and that the returned object from each request was the value of the result.

Asynchronous Example

The following code illustrates how to send requests asynchronously. Before you can run this example on your own computer make sure to Start the Network Manager and start the BasicMath Service.

# asynchronous.py
#
# This script takes about 6 seconds to run.

import time
from msl.network import connect

# Connect to the Manager (that is running on the same computer)
cxn = connect()

# Establish a link to the BasicMath Service
bm = cxn.link('BasicMath')

# Get the start time before sending the requests
t0 = time.perf_counter()

# Create asynchronous requests by using the asynchronous=True keyword argument
# The returned object is a Future object (not the result of each request)
add = bm.add(1, 2, asynchronous=True)
subtract = bm.subtract(1, 2, asynchronous=True)
multiply = bm.multiply(1, 2, asynchronous=True)
divide = bm.divide(1, 2, asynchronous=True)
is_positive = bm.ensure_positive(1, asynchronous=True)
power = bm.power(2, 4, asynchronous=True)

# There are different ways to gather the results of the Future objects.
# Calling result() on the Future will block until the result becomes
# available (or until the request raised an exception). Note, the
# result() method also supports a timeout argument. You can also
# register callbacks to be called when a Future is done.

# Print the results
print(f'1+2= {add.result()}')
print(f'1-2= {subtract.result()}')
print(f'1*2= {multiply.result()}')
print(f'1/2= {divide.result()}')
print(f'is positive? {is_positive.result()}')
print(f'2**4= {power.result()}')

# The total time that passed to receive all results
dt = time.perf_counter() - t0
print(f'Total execution time: {dt:.2f} seconds')

# Disconnect from the Manager
cxn.disconnect()

The output of the asynchronous.py program will be:

1+2= 3
1-2= -1
1*2= 2
1/2= 0.5
is positive? True
2**4= 16
Total execution time: 6.02 seconds

The Total execution time value will be slightly different for you, but the important thing to notice is that executing all requests took about 6 seconds (i.e., max(1, 2, 3, 4, 5, 6) for the time.sleep() functions in the BasicMath Service) and that the returned object from each request was a Future instance which we needed to get the result() of.

Synchronous vs Asynchronous comparison

Comparing the total execution time for the Synchronous Example and the Asynchronous Example we see that the asynchronous program is 3.5 times faster. Choosing whether to send a request synchronously or asynchronously is performed by passing in an asynchronous=False or asynchronous=True keyword argument, respectively. Also, in the synchronous example when a request is sent the object that is returned is the result of the method from the BasicMath Service, whereas in the asynchronous example the returned value is a Future object that provides the result later.

Synchronous

Asynchronous

Total execution time

21 seconds

6 seconds

Keyword argument to invoke

asynchronous=False (default)

asynchronous=True

Returned value from request

the result

a Future object