Best Practice Of Python S Project Structure

There's no standard answer about how to structure the python's project.I'll introduce one relatively reasonable solution.Here's the example.

.
├── ChangeLog
├── README.md
├── build.sh
├── build_and_install_egg.sh
├── clean_build.sh
├── docker
│   ├── DockerfileTemplate
│   ├── docker-compose-template.yaml
│   ├── env.sh
│   └── start.sh
├── setup.py
├── src
│   └── salvage
│       ├── __init__.py
│       ├── configuration
│       ├── controller
│       ├── core
│       ├── main.py
│       ├── model
│       ├── repository
│       ├── resource
│       ├── service
│       └── util
└── test
    └── salvage
        └── configuration

1.Chnagelog

Changelog is very important for a project.It records the release history of the project and helps the developer find bugs quickly.Here is the recommended example of a Changelog.

[1.0.0]
* first release

[0.2.1]
* fix the deadlock problem

[0.2.0]
* add multiprocessing and improve the performance

[0.1.0]
* first commit

Normally,we often use three numbers to represent the version number.The first number tracks major changes,the second tracks minor changes,the last tracks patches or bug fix.

2.README

README or README.md is used to introduce the basic information of the project,including :

• the purpose of the project

• how to run the project

• how to develop the project

• introduce the structure of the code

3.build script

Python use wheel or egg to pack the code,we can use build script to reduce the redundant typing work.Here's the example.

#!/bin/bash

python3 setup.py bdist_egg bdist_wheel

We can also use a clean_build.sh to clean the build products.

#!/bin/sh

if [ -d build ]; then
  rm -dr build
fi
if [ -d dist ]; then
  rm -dr dist
fi
if [ -d src/salvage.egg-info ]; then
  rm -dr src/salvage.egg-info
fi

You maybe want to build and install with easy install.

#!/bin/bash

source build.sh
sudo easy_install dist/salvage-*-py3.8.egg
source clean_build.sh

Cleaning the build products is recommended because it will influence the navigate operation of some IDE.You may modify the built python code ,not the project files.Clean save your time!

4.setup.py

We always use setuptools as the package tools.The documentation of setuptools is here.I'll only give some instructions about the basic usage of this tool base on one example.

#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
import os
import setuptools

def find_version(file_name):
    with open(file_name, encoding='utf-8') as file_handle:
        lines = file_handle.readlines()
        latest_version = lines[0].strip(os.linesep).rstrip(']').lstrip('[')
        print("Salvage:", latest_version)
        return latest_version

setuptools.setup(
    name='salvage',
    version=find_version("./ChangeLog"),
    description='toolbox',
    long_description='operation_toolbox',
    long_description_content_type='text/markdown',
    # packages=['salvage'],
    packages=setuptools.find_packages(where='src', include=['salvage*', ]),
    package_dir={'': 'src'},
    install_requires=[
        'requests',
        'pymysql',
        'wisbec',
        'bottle',
        'bottle-redis',
        'sqlalchemy',
        'sqlalchemy_utils',
        'tldextract',
        'redis'
    ],
    package_data={
        '': [
            'www/*',
            'config/*'
        ]
    },
    include_package_data=True,
    entry_points={
        "console_scripts": [
            "salvage = salvage.main:main",
        ]
    },
    python_requires='>=3.7,<3.9',
    platforms=['any'],
    classifiers=[
        'Development Status :: 5 - Production/Stable',
        'Intended Audience :: Developers',
        'License :: OSI Approved :: MIT License',
        'Operating System :: Unix',
        'Operating System :: POSIX',
        'Operating System :: Microsoft :: Windows',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: 3.8',
        'Programming Language :: Python :: Implementation :: CPython',
        'Programming Language :: Python :: Implementation :: PyPy',
        'Topic :: Utilities',
    ],
    zip_safe=False)

setuptools use setup method to make the package.We just need to specify some parameters.Here's the meaning of some important arguments.

• name:the name of the project

• version:the version of the project ,you can specify it each time you make changes to your project.I recommend use a simple function find_version to read the version number from your Changelog file.You can just cpoy the function to your setup.py ,feel free to use it😏!

• description:the description of your project .

• long_description:similar to description ,not important.

• packages:this parameter is used to specify which Python packages you want to pack.Usually ,we use setuptools.find_packages to find the Python packages.where means the package location ,include means package name .You can use the shell blob to represent your Python package name.Here the salvage* means all the Python packages which has salvage as the prefix .For example ,salvagecore and so on.You can also use exclude to exclude some Python packages which you do not want to include in the package.

• package_dir :the : in the example means root directory ,src means the Python package dir.

• install_requires:the dependency of the project.easy_install and pip will automatically install these packges you specifed .

• package_data:the resource file you want to pack .Sometimes you need to add some non-python file in your package.The '' in the example means root directory.**Attention,these means the package root.setuptools will look up in all of your target packages' directories.Here is the files in src :

  .
  └── salvage
      ├── __init__.py
      ├── main.py
      ├── resource
      │   ├── __init__.py
      │   ├── config
      │   │   ├── dev.ini
      │   │   └── release.ini
      │   ├── runtime.py
      │   └── www
      │       ├── adb.bundle.js
      │       └── index.html
      └── util
          └── __init__.py

  the www/* means pack all files in www directory ,so adb.bundle.js and index.html will be include in the package.

• include_package_data:if this parameter is True ,the package_data make sense .

• entry_points:this argument specify the entry point of your package.console_scripts means you can run the code from console after you installed the package.salvage = salvage.main:main means the console script name is salvage and the script entry point is the main function in salvage package's main.py file.Python will generate a script file on the PATH ,which is usually /usr/local/bin.

• python_requires:this stipulated the ranges the Python version of your packages.>=3.7,<3.9 means you can use this package with Python 3.7/3.8/3.9.

• platform:this means the platform of your package ,any means all platform is compatible .

5.code structure

I recommend use src to place all of your code and test to place your test code.You can also use the project name to replace src.