mrjob.local - run locally for testing

class mrjob.local.LocalMRJobRunner(**kwargs)

Runs an MRJob locally, for testing purposes.

This is the default way of running jobs; we assume you’ll spend some time debugging your job before you’re ready to run it on EMR or Hadoop.

It’s rare to need to instantiate this class directly (see __init__() for details).

LocalMRJobRunner simulates the following jobconf variables:

• mapreduce.job.cache.archives
• mapreduce.job.cache.files
• mapreduce.job.cache.local.archives
• mapreduce.job.cache.local.files
• mapreduce.job.id
• mapreduce.job.local.dir
• mapreduce.map.input.file
• mapreduce.map.input.length
• mapreduce.map.input.start
• mapreduce.task.attempt.id
• mapreduce.task.id
• mapreduce.task.ismap
• mapreduce.task.output.dir
• mapreduce.task.partition

LocalMRJobRunner adds the current working directory to the subprocesses’ PYTHONPATH, so if you’re using it to test an EMR job locally, be aware that it may see more Python modules than will actaully be uploaded. This behavior may change in the future.