Python zipapp Review
Published 2023-05-31
Python's zipapp module provides a convenient means to package pure-python modules into a single "binary".
Introduction
Maybe you've done this?
- I have script to share
- It has classes and functions that I've neatly componentized
- I just want to give someone a single file to run
- I copy/paste all the components into the file and give it to my colleague
- Rinse, repeat everytime I make a change
Python's zipapp module can really help simplify this process. What follows is the blogification of a talk I gave at work.
Trivial Example
Some python (I'm saving the file as __main__.py – ugly but necessary for now):
def main(): print("Hello, world!") if __name__ == "__main__": main()
That can be copied to your friend, then as long as they have a recent version of Python (3!):
python3 __main__.py
Hello, world!
Trivial… and weird
Yep, trivial, but I can also put it in a zipfile:
zip hello-world.zip __main__.py
adding: __main__.py (deflated 13%)
Run it!
python3 hello-world.zip
Hello, world!
Trivial… and weirder
What's in the file?
unzip -l hello-world.zip
Archive: hello-world.zip Length Date Time Name --------- ---------- ----- ---- 78 2023-05-31 22:46 __main__.py --------- ------- 78 1 fileNow this is fun:
echo "#!/usr/bin/env python3" | \ cat - hello-world.zip > hello-world-run.zip chmod 0755 hello-world-run.zip
And run it!
./hello-world-run.zip
Hello, world!
Python now has a module to help: zipapp
Build it with zipapp (new in Python 3.5):
python3 -m zipapp trivial/ -o /tmp/hello-world-zipapp \ -p "/usr/bin/env python3"
- I'm setting the shebang line – Linux and (maybe) Mac specific
- You can still run it by typing:
python3 hello-worldif you want to avoid setting it or if the setting is incorrect for your environment
- You can still run it by typing:
- I'm setting the shebang line – Linux and (maybe) Mac specific
The program is ready to run:
python3 /tmp/hello-world-zipapp
Hello, world!
But I have a local module I want to import
How about pulling the launch schedule from Spaceflight Now?
curl -s https://spaceflightnow.com/feed/ | grep '<title>' | head -3<title>Spaceflight Now</title> <title>Private astronauts splash down to close out 9-day commercial research mission</title> <title>Live coverage: SpaceX launches more Starlink satellites from California</title>
Here's a module to retrieve that RSS feed and pull out the interesting bits - I'll save it as rss.py:
import xml.etree.ElementTree as ET import requests class RetrieveRSS: def __init__(self, feed): self.feed = feed self.results = list() def get(self, raw_rss=None): raw_rss = raw_rss or requests.get(self.feed).text root = ET.fromstring(raw_rss) rss_links = dict() for channel in root.findall("channel"): channel_title = channel.find("title").text rss_links[channel_title] = list() for channel_item in channel.findall("item"): rss_links[channel_title].append( (channel_item.find("title").text, channel_item.find("link").text)) return rss_links
And we need a main (and some Python path magic) saved as get_rss_links.py:
import os import sys sys.path.append(os.path.dirname( os.path.realpath(__file__))) from rss import RetrieveRSS def main(): feed_url = sys.argv[1] feed = RetrieveRSS(feed_url) rss_links = feed.get() for channel in rss_links: for link in rss_links[channel]: print(f"Title: {link[0]}") print(f" Link: {link[1]}")
We can build a package with (notice "-m" option):
python3 -m zipapp with_modules/ -o /tmp/get-feeds \ -p "/usr/bin/env python3" \ -m "get_rss_links:main"
Does it work?
/tmp/get-feeds 'https://spaceflightnow.com/feed/' | head -6Title: Private astronauts splash down to close out 9-day commercial research mission Link: https://spaceflightnow.com/2023/05/31/private-astronauts-splash-down-to-close-out-9-day-commercial-research-mission/ Title: Live coverage: SpaceX launches more Starlink satellites from California Link: https://spaceflightnow.com/2023/05/30/falcon-9-starlink-2-10-mission-status-center/ Title: Live coverage: U.S.-Saudi commercial astronaut crew returns to Earth Link: https://spaceflightnow.com/2023/05/30/ax-2-return-mission-status-center/
What does the zip file look like?
unzip -l /tmp/get-feeds
Archive: /tmp/get-feeds Length Date Time Name --------- ---------- ----- ---- 368 2023-05-31 22:48 get_rss_links.py 695 2023-05-31 22:47 rss.py 66 2023-05-31 22:48 __main__.py --------- ------- 1129 3 filesThe zipapp module creates __main__.py:
unzip -p /tmp/get-feeds __main__.py
# -*- coding: utf-8 -*- import get_rss_links get_rss_links.main()
Importing from PyPI
Just like the previous example, but you can bring in a module from PyPI. Here's the requirements.txt:
cowsay==4.0Main program, sayit.py:
import sys import cowsay def main(): cowsay.cow(sys.argv[1])Build the package in a Python virtual environment:
python3 -m venv venv
python3 -m pip install --upgrade -r requirements.txt \ --target venvcp sayit.py venv/
python3 -m zipapp venv -p "/usr/bin/env python3" \ -o /tmp/sayit -m "sayit:main"
Test:
/tmp/sayit "This works on `date`!"___________________________________________ | This works on Wed May 31 22:48:56 PDT 2023! | =========================================== \ \ ^__^ (oo)\_______ (__)\ )\/\ ||----w | || ||
Support python2 and python3 in one package
I have come code that has to run under Python 2 and 3.
I have code for each version of Python:
ls -1 sender/
README.md __init__.py __main__.py __pycache__ main2.py main3.py sender.py sender2.py
Here's what the main code looks like:
import os import sys sys.path.append(os.path.dirname( os.path.realpath(__file__))) if __name__ == "__main__": if sys.version_info[0] == 2: from main2 import run else: from main3 import run run() sys.exit(0)
The Gotchas
- Pure Python only packaging
- Favorite modules: numpy, pandas, scikit-learn
- Have C-modules, which are not transportable (or at least I have yet to figure out the mechanism)
- One workaround that I haven't tried: shiv
Popular modules, so maybe you don't have to include them…
python3 -m venv --system-site-packages venv
- Now you have access to the any Python packages installed on the system
org-mode