Python and Kafka with Quix Streams

Supporting code for our step-by-step coding walkthroughs.

A Simple Producer

There's a complete walkthrough video here. To run the code in this repo:

# Set up an environment.
cd send_to_kafka
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

# Run the producer.
python3 main.py

A Simple Consumer

There's a complete walkthrough video here. To run the code in this repo:

# Set up an environment.
cd read_from_kafka
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

# Run the consumer.
python3 main.py

A Simple Processor

There's a complete walkthrough video here. To run the code in this repo:

# Set up an environment.
cd weather_processor
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

# Run the stream processor.
python3 main.py

Stream Processing to Google Spreadsheets

There's a complete walkthrough video here.

To run the code in this repo, you'll first need to create a Google Developer client API key file, using the Google Developer Console. Copy that client_secret.json file it gives you into the weather_to_google directory. Then:

# Set up an environment.
cd weather_to_google
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

# Run the stream processor.
python3 main.py

Performant Python Producers

There's a complete walkthrough video here.

# Set up an environment.
cd github-firehose
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt

# Run the stream processor.
python3 main.py