atmo.settings

Django settings for atmo project.

For more information on this file, see https://docs.djangoproject.com/en/1.9/topics/settings/

For the full list of settings and their values, see https://docs.djangoproject.com/en/1.9/ref/settings/

class atmo.settings.AWS

AWS settings

AWS_CONFIG = {'ACCOUNTING_APP_TAG': 'telemetry-analysis', 'ACCOUNTING_TYPE_TAG': 'worker', 'AWS_REGION': 'us-west-2', 'CODE_BUCKET': 'telemetry-analysis-code-2', 'EC2_KEY_NAME': '20161025-dataops-dev', 'EMAIL_SOURCE': 'telemetry-alerts@mozilla.com', 'INSTANCE_APP_TAG': 'telemetry-analysis-worker-instance', 'LOG_BUCKET': 'telemetry-analysis-logs-2', 'MASTER_INSTANCE_TYPE': 'c3.4xlarge', 'MAX_CLUSTER_LIFETIME': 24, 'MAX_CLUSTER_SIZE': 30, 'PRIVATE_DATA_BUCKET': 'telemetry-private-analysis-2', 'PUBLIC_DATA_BUCKET': 'telemetry-public-analysis-2', 'WORKER_INSTANCE_TYPE': 'c3.4xlarge'}

The AWS config values.

PUBLIC_DATA_URL = 'https://s3-us-west-2.amazonaws.com/telemetry-public-analysis-2/'

The URL of the S3 bucket with public job results.

PUBLIC_NB_URL = 'https://nbviewer.jupyter.org/url/s3-us-west-2.amazonaws.com/telemetry-public-analysis-2/'

The URL to show public Jupyter job results with.

class atmo.settings.Base

Configuration that may change per-environment, some with defaults.

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

SITE_URL = 'http://localhost:8000'

The URL under which this instance is running

class atmo.settings.Build

Configuration to be used in build (!) environment

CONSTANCE_CONFIG

Dictionary that remembers insertion order

DATABASES

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

class atmo.settings.CSP

CSP settings

class atmo.settings.Celery

The Celery specific Django settings.

CELERY_BEAT_MAX_LOOP_INTERVAL = 5

redbeat likes fast loops

CELERY_BEAT_SCHEDULE = {'clean_orphan_obj_perms': {'schedule': <crontab: 30 3 * * * (m/h/d/dM/MY)>, 'task': 'atmo.tasks.cleanup_permissions'}, 'deactivate_clusters': {'schedule': <crontab: * * * * * (m/h/d/dM/MY)>, 'options': {'expires': 40, 'soft_time_limit': 15}, 'task': 'atmo.clusters.tasks.deactivate_clusters'}, 'expire_jobs': {'schedule': <crontab: * * * * * (m/h/d/dM/MY)>, 'options': {'expires': 40, 'soft_time_limit': 15}, 'task': 'atmo.jobs.tasks.expire_jobs'}, 'send_expiration_mails': {'schedule': <crontab: */5 * * * * (m/h/d/dM/MY)>, 'options': {'expires': 240}, 'task': 'atmo.clusters.tasks.send_expiration_mails'}, 'send_run_alert_mails': {'schedule': <crontab: * * * * * (m/h/d/dM/MY)>, 'options': {'expires': 40}, 'task': 'atmo.jobs.tasks.send_run_alert_mails'}, 'update_clusters': {'schedule': <crontab: */5 * * * * (m/h/d/dM/MY)>, 'options': {'expires': 180, 'soft_time_limit': 270}, 'task': 'atmo.clusters.tasks.update_clusters'}, 'update_jobs_statuses': {'schedule': <crontab: */15 * * * * (m/h/d/dM/MY)>, 'options': {'expires': 600, 'soft_time_limit': 870}, 'task': 'atmo.jobs.tasks.update_jobs_statuses'}}

The default/initial schedule to use.

CELERY_BEAT_SCHEDULER = 'redbeat.RedBeatScheduler'

The scheduler to use for periodic and scheduled tasks.

CELERY_BROKER_TRANSPORT_OPTIONS = {'fanout_patterns': True, 'fanout_prefix': True, 'visibility_timeout': 691200}

The Celery broker transport options

CELERY_REDBEAT_LOCK_TIMEOUT = 25

Unless refreshed the lock will expire after this time

CELERY_RESULT_BACKEND = 'django-db'

Use the django_celery_results database backend.

CELERY_RESULT_EXPIRES = datetime.timedelta(14)

Throw away task results after two weeks, for debugging purposes.

CELERY_TASK_SEND_SENT_EVENT = True

Send SENT events as well to know when the task has left the scheduler.

CELERY_TASK_SOFT_TIME_LIMIT = 300

Add a 5 minute soft timeout to all Celery tasks.

CELERY_TASK_TIME_LIMIT = 600

And a 10 minute hard timeout.

CELERY_TASK_TRACK_STARTED = True

Track if a task has been started, not only pending etc.

CELERY_WORKER_DISABLE_RATE_LIMITS = True

Completely disable the rate limiting feature since it’s costly

CELERY_WORKER_HIJACK_ROOT_LOGGER = False

Stop hijacking the root logger so Sentry works.

class atmo.settings.Constance

Constance settings

CONSTANCE_ADDITIONAL_FIELDS = {'announcement_styles': [<class 'django.forms.fields.ChoiceField'>, {'widget': <django.forms.widgets.Select object at 0x7f3b68f88be0>, 'choices': (('success', 'success (green)'), ('info', 'info (blue)'), ('warning', 'warning (yellow)'), ('danger', 'danger (red)'))}], 'announcement_title': [<class 'django.forms.fields.CharField'>, {'widget': <django.forms.widgets.TextInput object at 0x7f3b68f88b70>}]}

Adds custom widget for announcements.

CONSTANCE_CONFIG = {'ANNOUNCEMENT_CONTENT': ('', 'The announcement content.'), 'ANNOUNCEMENT_CONTENT_MARKDOWN': (False, 'Whether the announcement content should be rendered as CommonMark (Markdown).'), 'ANNOUNCEMENT_ENABLED': (False, 'Whether to show the announcement on every page.'), 'ANNOUNCEMENT_TITLE': ('Announcement', 'The announcement title.', 'announcement_title'), 'ANNOUNCMENT_STYLE': ('info', 'The style of the announcement.', 'announcement_styles'), 'AWS_EFS_DNS': ('fs-616ca0c8.efs.us-west-2.amazonaws.com', 'The DNS name of the EFS mount for EMR clusters'), 'AWS_SPARK_EMR_BUCKET': ('telemetry-spark-emr-2-stage', 'The S3 bucket where the EMR bootstrap scripts are located'), 'AWS_SPARK_INSTANCE_PROFILE': ('telemetry-spark-cloudformation-stage-TelemetrySparkInstanceProfile-UCLC2TTGVX96', 'The AWS instance profile to use for the clusters'), 'AWS_SPOT_BID_CORE': (0.84, 'The spot instance bid price for the cluster workers'), 'AWS_USE_SPOT_INSTANCES': (True, 'Whether to use spot instances on AWS')}

The default config values.

CONSTANCE_CONFIG_FIELDSETS = {'AWS': ('AWS_USE_SPOT_INSTANCES', 'AWS_SPOT_BID_CORE', 'AWS_EFS_DNS', 'AWS_SPARK_EMR_BUCKET', 'AWS_SPARK_INSTANCE_PROFILE'), 'Announcements': ('ANNOUNCEMENT_ENABLED', 'ANNOUNCMENT_STYLE', 'ANNOUNCEMENT_TITLE', 'ANNOUNCEMENT_CONTENT', 'ANNOUNCEMENT_CONTENT_MARKDOWN')}

Some fieldsets for the config values.

CONSTANCE_REDIS_CONNECTION_CLASS = 'django_redis.get_redis_connection'

Using the django-redis connection function for the backend.

class atmo.settings.Core

Configuration that will never change per-environment.

BASE_DIR = '/home/docs/checkouts/readthedocs.org/user_builds/atmo/envs/latest/lib/python3.5/site-packages/mozilla_atmo-2018.4.1.dev65+gd62d747-py3.5.egg'

Build paths inside the project like this: os.path.join(BASE_DIR, …)

INSTALLED_APPS = ['atmo.apps.AtmoAppConfig', 'atmo.clusters', 'atmo.jobs', 'atmo.apps.KeysAppConfig', 'atmo.users', 'atmo.stats', 'guardian', 'constance', 'constance.backends.database', 'dockerflow.django', 'django_celery_monitor', 'django_celery_results', 'flat_responsive', 'django.contrib.sites', 'django.contrib.admin', 'django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'mozilla_django_oidc']

The installed apps.

SITE_ID = 1

Using the default first site found by django.contrib.sites

THIS_DIR = '/home/docs/checkouts/readthedocs.org/user_builds/atmo/envs/latest/lib/python3.5/site-packages/mozilla_atmo-2018.4.1.dev65+gd62d747-py3.5.egg/atmo'

The directory in which the settings file reside.

VERSION = None

The current ATMO version.

class atmo.settings.Dev

Configuration to be used during development and base class for testing

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

class atmo.settings.Docs

Configuration to be used in the documentation environment

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

class atmo.settings.Prod

Configuration to be used in prod environment

CONSTANCE_CONFIG

Dictionary that remembers insertion order

DATABASES

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

class atmo.settings.Stage

Configuration to be used in stage environment

DATABASES

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)

class atmo.settings.Test

Configuration to be used during testing

LOGGING()

dict() -> new empty dictionary dict(mapping) -> new dictionary initialized from a mapping object’s

(key, value) pairs

dict(iterable) -> new dictionary initialized as if via:

d = {} for k, v in iterable:

d[k] = v

dict(**kwargs) -> new dictionary initialized with the name=value pairs

in the keyword argument list. For example: dict(one=1, two=2)