English
Skip to page content Skip to main navigation

e.g. "templates" or "datastore"

# Uploading and Managing a Python App

The App Engine Python SDK includes a command for interacting with App Engine named `appcfg.py`. You can use this command to upload new versions of the code, configuration and static files for your app to App Engine. You can also use the command to manage datastore indexes and download log data.

- Uploading the App
- Updating Indexes
- Deleting Unused Indexes
- Managing Scheduled Tasks
- Downloading Logs
- Uploading Data
- Using an HTTP Proxy
- Command-Line Arguments

## Uploading the App

To upload application files, run the `appcfg.py` command with the `update` action and the name of your application's root directory. The root directory should contain the app.yaml file for the application.

    appcfg.py update myapp/

`appcfg.py` gets the application ID from the `app.yaml` file, and prompts you for the email address and password of your Google account. After successfully signing in with your account, `appcfg.py` stores a "cookie" so that it does not need to prompt for a password on subsequent attempts.

You can specify the email address on the command line using the `--email` option. You cannot specify the password as a command line option.

    appcfg.py --email=Albert.Johnson@example.com update myapp/

## Updating Indexes

When you upload an application using `appcfg.py update`, the update includes the app's `index.yaml` file. If the `index.yaml` file defines an index that doesn't exist yet on App Engine, App Engine creates the new index. Depending on how much data is already in the datastore that needs to be mentioned in the new index, the process of creating the index may take a while. If the app performs a query that requires an index that hasn't finished building yet, the query will raise an exception.

To prevent this, you must ensure that the new version of the app that requires a new index is not the live version of the application until the indexes finish building. One way to do this is to give the app a new version number in `app.yaml` whenever you add or change an index in `index.yaml`. The app is uploaded as a new version, and does not become the default version automatically. When your indexes have finished building, you change the default version to the new one using the "Versions" section of the Admin Console.

Another way to ensure that new indexes are built before the new app goes live is to upload the `index.yaml` configuration separately before uploading the app. To upload only the index configuration for an app, use the following command:

    appcfg.py update_indexes myapp/

You can check the status of the app's indexes from the "Indexes" section of the Admin Console.

## Deleting Unused Indexes

When you change or remove an index from `index.yaml`, the original index is not deleted from App Engine automatically. This gives you the opportunity to leave an older version of the app running while new indexes are being built, or to revert to the older version immediately if a problem is discovered with a newer version.

When you are sure that old indexes are no longer needed, you can delete them from App Engine using the following command:

    appcfg.py vacuum_indexes myapp/

This command deletes all indexes for the app that are not mentioned in the most recently uploaded version of `index.yaml`.

## Managing Scheduled Tasks

App Engine supports scheduled tasks (known as cron jobs). You specify these in a file called `cron.yaml`, and upload them using the `appcfg.py update_cron` command:

    appcfg.py update_cron myapp/

`appcfg update` will also upload cron job specifications if the file exists. For more on cron jobs, see the Cron Jobs documentation.

## Downloading Logs

App Engine maintains a log of messages that your application emits using the `logging` module from the Python standard library, as well as other messages printed to the standard error stream. App Engine also records each request in the log. You can browse your app's logs from the "Logs" section of the Admin Console.

If you wish to perform more detailed analysis of your application's logs, you can download the log data to a file on your computer. To download logs to a file named `mylogs.txt`, use the following command:

    appcfg.py request_logs myapp/ mylogs.txt

By default, the command downloads log messages from the current calendar day (since midnight Pacific Time) with a log level of INFO or higher (omitting DEBUG level messages). The command overwrites the local log file. You can adjust the number of days, the minimum log level, and whether to overwrite or append to the local log file using command-line options. See below for more information on these options.

You can limit the log messages that are downloaded to just those emitted during request on a given domain name using the `--vhost=...` option. You can use this to download the logs for your live app using a Google Apps domain or `app-id.appspot.com`, excluding log messages emitted by versions you are testing on URLs such as `2.latest.app-id.appspot.com`. Or you can use it to download just the log messages for a given test domain.

## Uploading Data

You can upload large amounts data to your application's data store using `appcfg.py` with the `upload_data` action. This action requires that you set up a special request handler in your Python application. For complete instructions on how to use this feature, see Uploading Data.

## Using an HTTP Proxy

If you are running `appcfg.py` behind an HTTP proxy, you must tell `appcfg.py` the name of the proxy. To set an HTTP proxy for `appcfg.py`, set the `http_proxy` environment variable.

Using Windows (in Command Prompt):

    set HTTP_PROXY=http://cache.mycompany.com:3128
    appcfg.py update myapp

Using the command line in Mac OS X (in Terminal) or Linux:

    export http_proxy="http://cache.mycompany.com:3128"
    appcfg.py update myapp

## Command-Line Arguments

The `appcfg.py` command takes a set of options, an action, and arguments for the action.

The following actions are available:

`appcfg.py [options] update <app-directory>`

  Uploads files for an application given the application's root directory. The application ID and version are taken from the `app.yaml` file in the app directory.

`appcfg.py [options] rollback <app-directory>`

  Undo a partially completed update for the given application. You can use this if an update was interrupted, and the command is reporting that the application cannot be updated due to a lock.

`appcfg.py [options] update_indexes <app-directory>`

  Updates datastore indexes in App Engine to include newly added indexes. If a new version of your application requires an additional index definition that was added to index.yaml, you can update your index definitions in App Engine prior to uploading the new version of your application. Running this action a few hours prior to uploading your new application version will give the indexes time to build and be serving when the application is deployed.

`appcfg.py [options] vacuum_indexes <app-directory>`

  Delete unused datastore indexes in App Engine. If an index definition is removed from index.yaml, the index is not deleted automatically when the application is uploaded because it may be in use by another version of the application. Run this action when all old indexes are no longer needed.

`appcfg.py [options] request_logs <app-directory> <output-file>`

  Retrieve log data for the application running on App Engine. `output-file` is the name of the file to create, replace or append (if the `--append` flag is set). If `output-file` is a hyphen (`-`), the log data is printed to the console. The following options apply to `request_logs`:

  `--num_days=...`

    The number of days of log data to retrieve, ending on the current date at midnight UTC. A value of 0 retrieves all available logs. If `--append` is given, then the default is 0, otherwise the default is 1.

  `--severity=...`

    The minimum log level for the log messages to retrieve. The value is a number corresponding to the log level: 4 for CRITICAL, 3 for ERROR, 2 for WARNING, 1 for INFO, 0 for DEBUG. All messages at the given log level and above will be retrieved. Default is 1 (INFO).

  `--append`

    Append the fetched data to the output file, starting with the first log line not already present in the file. Running this command once a day with `--append` results in a file containing all log data.

    The default is to overwrite the output file. Does not apply if `output-file` is `-` (printing to the console).

  `--vhost=...`

    If present, limits the log messages downloaded to just those emitted by requests for a given domain name. For instance, `--vhost=example.com` will download just the log messages for the live app at the Google Apps domain `example.com`, excluding any log messages emitted by a new version being tested at `2.latest.myapp.appspot.com`. Similarly, `--vhost=2.latest.myapp.appspot.com` downloads just the logs for the test version, excluding the live version.

    The value is a regular expression that matches the `Host` header of the incoming requests. Note that the pattern is case sensitive, even though domain names usually are not.

`appcfg.py [options] upload_data <app-directory>`

  Upload data to the datastore from CSV files. For complete information on using this feature, see Uploading Data.

`appcfg.py help <action>`

  Print a help message about the given action, then quit.

The `appcfg.py` command accepts the following options for all actions:

`--quiet`

  Do not print messages when successful.

`--verbose`

  Print messages about what the command is doing.

`--noisy`

  Print many messages about what the command is doing. This is mostly useful when working with the App Engine team to troubleshoot an upload issue.

`--email=...`

  The email address of the Google account of an administrator for the application, for actions that require signing in. If omitted and no cookie is stored from a previous use of the command, the command will prompt for this value.

`--passin`

  If given, the tool accepts the Google account password in stdin instead of prompting for it interactively. This allows you to invoke the tool from a script without putting your password on the command line.

`--server=...`

  The App Engine server hostname. The default is `appengine.google.com`.

`--host=...`

  The hostname of the local machine for use with remote procedure calls.

`--no_cookies`

  Do not store the administrator sign-in credentials as a cookie; prompt for a password every time.

`--force`

  Force deletion of unused indexes. By default, uploading an app does not delete unused indexes from the server, even if they do not appear in the `index.yaml` file.

`--max_size=...`

  A maximum size of files to upload, as a number of bytes. Files larger than this size will not be uploaded. The default is `1048576`. The server currently enforces a maximum file size of 1,048,576 bytes, so increasing this value will not have any effect.