Comparing Inter-process Communication Methods in Python

Tag: Engineering

Published on: 02 Jul 2021


Out of curiosity, I performed a simple performance benchmark of the different inter process communication methods between a Python and a C++ program.

The test is performed by recording the round-trip time of sending + receiving fixed-sized data chunks for a fixed number of times:

Python(send) --> C++ (receive) --> C++ (send) --> Python(receive, chk)

Benchmark Results

Named Pipe / FIFO

Data Chunk Size Iterations Total Time (Roundtrip) MB / s
1 KB 10000 570.143ms 35.08
500 KB 10000 14195.036 ms 704.46
1 MB 1000 5161.226 ms 387.50
10 MB 100 69419.094 ms 28.82

Memory-mapped File

Data Chunk Size Iterations Total Time (Roundtrip) MB / s
1 KB 100 3724.573ms 0.06
500 KB 100 4435.349 ms 22.54
1 MB 100 4416.842 ms 45.28
10 MB 100 9196.514 ms 217.48

Shared Memory

Data Chunk Size Iterations Total Time (Roundtrip) MB / s
1 KB 100000 190.588 ms 1049.38
500 KB 100000 5841.126 ms 17119.98
1 MB 100000 11978.522 ms 16696.54
10 MB 10000 17031.915 ms 11742.66

Socket

Data Chunk Size Iterations Total Time (Roundtrip) MB / s
1 KB 10000 263.460 ms 75.90
500 KB 10000 1642.042 ms 6089.98
1 MB 10000 3514.859 ms 5690.12
10 MB 1000 12740.363 ms 1569.82

References


© Chengxin Wang. All rights reserved.