# TORinator
A Tor IP address indexer. A local database is maintained and regularly updated
from multiple sources. By default, the sources are scraped every 45 minutes in
order to prevent rate limiting / blocking. You can override this using an
enviornment variable - details below.


## API Docs

    # Get All IPs
    curl -s http://127.0.0.1:5000/api/v1/ips | jq '.'

    # Get All Exclusions
    curl -s http://127.0.0.1:5000/api/v1/exclusions | jq '.'

    # Add Exclusion IP
    curl -X POST http://127.0.0.1:5000/api/v1/exclusions \
      -H 'Content-Type: application/json' \
      -d '{ "ip": "202.59.9.238" }' | jq '.'

    # Remove Exclusion IP
    curl -X DELETE http://127.0.0.1:5000/api/v1/exclusions/202.59.9.238


## Configuration
The application can be configured using environment variables. The following
vars are available:

| Environment Variable     | Description                     | Default |
|--------------------------|---------------------------------|---------|
| TORINATOR_DB_TYPE        | Database type (memory / sqlite) | sqlite  |
| TORINATOR_DATA_PATH      | Path to store the database      | ./      |
| TORINATOR_UPDATE_SOURCE  | Update source (web / file )     | web     |
| TORINATOR_UPDATE_CADENCE | Update cadence in seconds       | 2700    |

You can find examples in the `docker-compose.yml` file.


## Running (Production)
```
# Be sure image is built
docker-compose build

# Run compose in background. Service accessible via localhost:5000
docker-compose up -d

# Alternatively, in virtualenv
torinator run
```

## Running (Development)
Once you have installed the setup.py and torinator[dev] dependencies, you can
run `pre-commit` to lint all server files:

```
pre-commit run --all-files
```

To run:
```
# Setup virtual env
python3 -m venv torinator_venv
. torinator_venv/bin/activate

# Link torinator to working directory & install dev dependencies
python setup.py develop easy_install torinator[dev]

# Can use to run the server
torinator run

# Or with env vars
TORINATOR_UPDATE_SOURCE=file torinator run
```

### Adding Sources
While a little cumbersome, you can add additional sources in the
`./src/torinator/list_updater.py` file. In the `__init__`, add additional
sources with their URL's, parsing function, and filepath (if desired). Using
a filepath allows you to reference a local file for parsing instead of hitting
the web everytime. This is useful during development to prevent rate limiting.

Local files should be stored in `./src/torinator/resources/*` and are copied by
the `MANIFEST.in` during build.