mozpack package¶
Subpackages¶
- mozpack.chrome package
- Submodules
- mozpack.chrome.flags module
- mozpack.chrome.manifest module
Manifest
ManifestBinaryComponent
ManifestCategory
ManifestChrome
ManifestComponent
ManifestContent
ManifestContract
ManifestEntry
ManifestEntryWithRelPath
ManifestInterfaces
ManifestLocale
ManifestMultiContent
ManifestOverlay
ManifestOverload
ManifestOverride
ManifestResource
ManifestSkin
ManifestStyle
is_manifest()
parse_manifest()
parse_manifest_line()
- Module contents
- mozpack.packager package
- Submodules
- mozpack.packager.formats module
- mozpack.packager.l10n module
- mozpack.packager.unpack module
- Module contents
Submodules¶
mozpack.archive module¶
- class mozpack.archive.HackedType¶
Bases:
bytes
- class mozpack.archive.TarInfo(name='')¶
Bases:
TarInfo
- chksum¶
Header checksum.
- devmajor¶
Device major number.
- devminor¶
Device minor number.
- gid¶
Group ID of the user who originally stored this member.
- gname¶
Group name.
- linkname¶
Name of the target file name, which is only present in TarInfo objects of type LNKTYPE and SYMTYPE.
- mode¶
Permission bits.
- mtime¶
Time of last modification.
- name¶
Name of the archive member.
- offset¶
The tar header starts here.
- offset_data¶
The file’s data starts here.
- pax_headers¶
A dictionary containing key-value pairs of an associated pax extended header.
- size¶
Size in bytes.
- sparse¶
Sparse member information.
- tarfile¶
- type¶
REGTYPE, AREGTYPE, LNKTYPE, SYMTYPE, DIRTYPE, FIFOTYPE, CONTTYPE, CHRTYPE, BLKTYPE, GNUTYPE_SPARSE.
- Type:
File type. type is usually one of these constants
- uid¶
User ID of the user who originally stored this member.
- uname¶
User name.
- mozpack.archive.create_tar_bz2_from_files(fp, files, compresslevel=9)¶
Create a tar.bz2 file deterministically from files.
This is a glorified wrapper around
create_tar_from_files
that adds bzip2 compression.This function is similar to
create_tar_gzip_from_files()
.
- mozpack.archive.create_tar_from_files(fp, files)¶
Create a tar file deterministically.
Receives a dict mapping names of files in the archive to local filesystem paths or
mozpack.files.BaseFile
instances.The files will be archived and written to the passed file handle opened for writing.
Only regular files can be written.
FUTURE accept a filename argument (or create APIs to write files)
- mozpack.archive.create_tar_gz_from_files(fp, files, filename=None, compresslevel=9)¶
Create a tar.gz file deterministically from files.
This is a glorified wrapper around
create_tar_from_files
that adds gzip compression.The passed file handle should be opened for writing in binary mode. When the function returns, all data has been written to the handle.
mozpack.copier module¶
- class mozpack.copier.FileCopier¶
Bases:
FileRegistry
FileRegistry with the ability to copy the registered files to a separate directory.
- copy(destination, skip_if_older=True, remove_unaccounted=True, remove_all_directory_symlinks=True, remove_empty_directories=True)¶
Copy all registered files to the given destination path. The given destination can be an existing directory, or not exist at all. It can’t be e.g. a file. The copy process acts a bit like rsync: files are not copied when they don’t need to (see mozpack.files for details on file.copy).
By default, files in the destination directory that aren’t registered are removed and empty directories are deleted. In addition, all directory symlinks in the destination directory are deleted: this is a conservative approach to ensure that we never accidently write files into a directory that is not the destination directory. In the worst case, we might have a directory symlink in the object directory to the source directory.
To disable removing of unregistered files, pass remove_unaccounted=False. To disable removing empty directories, pass remove_empty_directories=False. In rare cases, you might want to maintain directory symlinks in the destination directory (at least those that are not required to be regular directories): pass remove_all_directory_symlinks=False. Exercise caution with this flag: you almost certainly do not want to preserve directory symlinks.
Returns a FileCopyResult that details what changed.
- class mozpack.copier.FileCopyResult¶
Bases:
object
Represents results of a FileCopier.copy operation.
- property existing_files_count¶
- property removed_directories_count¶
- property removed_files_count¶
- property updated_files_count¶
- class mozpack.copier.FileRegistry¶
Bases:
object
Generic container to keep track of a set of BaseFile instances. It preserves the order under which the files are added, but doesn’t keep track of empty directories (directories are not stored at all). The paths associated with the BaseFile instances are relative to an unspecified (virtual) root directory.
registry = FileRegistry() registry.add(‘foo/bar’, file_instance)
- add(path, content)¶
Add a BaseFile instance to the container, under the given path.
- contains(pattern)¶
Return whether the container contains paths matching the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.
- input_to_outputs_tree()¶
Return a dictionary mapping each input path to the set of impacted output paths.
All paths are normalized.
- match(pattern)¶
Return the list of paths, stored in the container, matching the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.
- output_to_inputs_tree()¶
Return a dictionary mapping each output path to the set of its required input paths.
All paths are normalized.
- paths()¶
Return all paths stored in the container, in the order they were added.
- remove(pattern)¶
Remove paths matching the given pattern from the container. See the mozpack.path.match documentation for a description of the handled patterns.
- required_directories()¶
Return the set of directories required by the paths in the container, in no particular order. The returned directories are relative to an unspecified (virtual) root directory (and do not include said root directory).
- class mozpack.copier.FileRegistrySubtree(base, registry)¶
Bases:
object
A proxy class to give access to a subtree of an existing FileRegistry.
Note this doesn’t implement the whole FileRegistry interface.
- add(path, content)¶
- contains(pattern)¶
- match(pattern)¶
- paths()¶
- remove(pattern)¶
- class mozpack.copier.Jarrer(compress=True)¶
Bases:
FileRegistry
,BaseFile
FileRegistry with the ability to copy and pack the registered files as a jar file. Also acts as a BaseFile instance, to be copied with a FileCopier.
- add(path, content, compress=None)¶
Add a BaseFile instance to the container, under the given path.
- copy(dest, skip_if_older=True)¶
Pack all registered files in the given destination jar. The given destination jar may be a path to jar file, or a Dest instance for a jar file. If the destination jar file exists, its (compressed) contents are used instead of the registered BaseFile instances when appropriate.
- open()¶
Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.
- preload(paths)¶
Add the given set of paths to the list of preloaded files. See mozpack.mozjar.JarWriter documentation for details on jar preloading.
mozpack.dmg module¶
- mozpack.dmg.chmod(dir)¶
Set permissions of DMG contents correctly
- mozpack.dmg.create_app_symlink(stagedir: Path, tmpdir: Path, hfs_tool: Path = None)¶
Make a symlink to /Applications. The symlink name is a space so we don’t have to localize it. The Applications folder icon will be shown in Finder, which should be clear enough for users.
- mozpack.dmg.create_dmg(source_directory: Path, output_dmg: Path, volume_name: str, extra_files: List[tuple], dmg_tool: Path, hfs_tool: Path, mkfshfs_tool: Path, attribution_sentinel: str = None)¶
Create a DMG disk image at the path output_dmg from source_directory.
Use volume_name as the disk image volume name, and use extra_files as a list of tuples of (filename, relative path) to copy into the disk image.
- mozpack.dmg.create_dmg_from_staged(stagedir: Path, output_dmg: Path, tmpdir: Path, volume_name: str, hfs_tool: Path = None, dmg_tool: Path = None, mkfshfs_tool: Path = None, attribution_sentinel: str = None)¶
Given a prepared directory stagedir, produce a DMG at output_dmg.
- mozpack.dmg.extract_dmg(dmgfile: Path, output: Path, dmg_tool: Path = None, hfs_tool: Path = None, dsstore: Path = None, icon: Path = None, background: Path = None)¶
- mozpack.dmg.extract_dmg_contents(dmgfile: Path, destdir: Path, dmg_tool: Path = None, hfs_tool: Path = None)¶
- mozpack.dmg.generate_hfs_file(stagedir: Path, tmpdir: Path, volume_name: str, mkfshfs_tool: Path)¶
When cross compiling, we zero fill an hfs file, that we will turn into a DMG. To do so we test the size of the staged dir, and add some slight padding to that.
- mozpack.dmg.rsync(source: Path, dest: Path)¶
rsync the contents of directory source into directory dest
- mozpack.dmg.set_folder_icon(dir: Path, tmpdir: Path, hfs_tool: Path = None)¶
Set HFS attributes of dir to use a custom icon
mozpack.errors module¶
- exception mozpack.errors.AccumulatedErrors¶
Bases:
Exception
Exception type raised from errors.accumulate()
- class mozpack.errors.ErrorCollector¶
Bases:
object
Error handling/logging class. A global instance, errors, is provided for convenience.
Warnings, errors and fatal errors may be logged by calls to the following functions:
errors.warn(message)
errors.error(message)
errors.fatal(message)
Warnings only send the message on the logging output, while errors and fatal errors send the message and throw an ErrorMessage exception. The exception, however, may be deferred. See further below.
- Errors may be ignored by calling:
errors.ignore_errors()
After calling that function, only fatal errors throw an exception.
The warnings, errors or fatal errors messages may be augmented with context information when a context is provided. Context is defined by a pair (filename, linenumber), and may be set with errors.context() used as a
context manager:
with errors.context(filename, linenumber): errors.warn(message)
Arbitrary nesting is supported, both for errors.context calls:
with errors.context(filename1, linenumber1): errors.warn(message) with errors.context(filename2, linenumber2): errors.warn(message)
as well as for function calls:
def func(): errors.warn(message) with errors.context(filename, linenumber): func()
Errors and fatal errors can have their exception thrown at a later time, allowing for several different errors to be reported at once before throwing. This is achieved with errors.accumulate() as a context manager:
with errors.accumulate(): if test1: errors.error(message1) if test2: errors.error(message2)
In such cases, a single AccumulatedErrors exception is thrown, but doesn’t contain information about the exceptions. The logged messages do.
- ERROR = 2¶
- FATAL = 3¶
- WARN = 1¶
- accumulate()¶
- context(file, line)¶
- property count¶
- error(msg)¶
- fatal(msg)¶
- get_context()¶
- ignore_errors(ignore=True)¶
- out = <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>¶
- warn(msg)¶
- exception mozpack.errors.ErrorMessage¶
Bases:
Exception
Exception type raised from errors.error() and errors.fatal()
mozpack.executables module¶
- mozpack.executables.elfhack(path)¶
Execute the elfhack command on the given path.
- mozpack.executables.get_type(path_or_fileobj)¶
Check the signature of the give file and returns what kind of executable matches.
- mozpack.executables.is_executable(path)¶
Return whether a given file path points to an executable or a library, where an executable or library is identified by: - the file extension on OS/2 and WINNT - the file signature on OS/X and ELF systems (GNU/Linux, Android, BSD, Solaris)
As this function is intended for use to choose between the ExecutableFile and File classes in FileFinder, and choosing ExecutableFile only matters on OS/2, OS/X, ELF and WINNT (in GCC build) systems, we don’t bother detecting other kind of executables.
- mozpack.executables.may_elfhack(path)¶
Return whether elfhack() should be called
- mozpack.executables.may_strip(path)¶
Return whether strip() should be called
- mozpack.executables.strip(path)¶
Execute the STRIP command with STRIP_FLAGS on the given path.
mozpack.files module¶
- class mozpack.files.AbsoluteSymlinkFile(path)¶
Bases:
File
File class that is copied by symlinking (if available).
This class only works if the target path is absolute.
- copy(dest, skip_if_older=True)¶
Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).
- class mozpack.files.BaseFile¶
Bases:
object
Base interface and helper for file copying. Derived class may implement their own copy function, or rely on BaseFile.copy using the open() member function and/or the path property.
- static any_newer(dest, inputs)¶
Compares the modification time of
dest
to multiple input files, and returns whether any of theinputs
is newer (has a later mtime) thandest
.
- copy(dest, skip_if_older=True)¶
Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).
- inputs()¶
Return an iterable of the input file paths that impact this output file.
- static is_older(first, second)¶
Compares the modification time of two files, and returns whether the
first
file is older than thesecond
file.
- property mode¶
Return the file’s unix mode, or None if it has no meaning.
- static normalize_mode(mode)¶
- open()¶
Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.
- read()¶
- size()¶
Returns size of the entry.
Derived classes are highly encouraged to override this with a more optimal implementation.
- class mozpack.files.BaseFinder(base, minify=False, minify_js=False, minify_js_verify_command=None)¶
Bases:
object
- contains(pattern)¶
Return whether some files under the base directory match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.
- find(pattern)¶
Yield path, BaseFile_instance pairs for all files under the base directory and its subdirectories that match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.
- get(path)¶
Obtain a single file.
Where
find
is tailored towards matching multiple files, this method is used for retrieving a single file. Use this method when performance is critical.Returns a
BaseFile
if at most one file exists orNone
otherwise.
- class mozpack.files.ComposedFinder(finders)¶
Bases:
BaseFinder
Composes multiple File Finders in some sort of virtual file system.
A ComposedFinder is initialized from a dictionary associating paths to *Finder instances.
Note this could be optimized to be smarter than getting all the files in advance.
- find(pattern)¶
Yield path, BaseFile_instance pairs for all files under the base directory and its subdirectories that match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.
- class mozpack.files.DeflatedFile(file)¶
Bases:
BaseFile
File class for members of a jar archive. DeflatedFile.copy() effectively extracts the file from the jar archive.
- open()¶
Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.
- class mozpack.files.Dest(path)¶
Bases:
object
- Helper interface for BaseFile.copy. The interface works as follows:
read() and write() can be used to sequentially read/write from the underlying file.
a call to read() after a write() will re-open the underlying file and read from it.
a call to write() after a read() will re-open the underlying file, emptying it, and write to it.
- close()¶
- exists()¶
- property name¶
- read(length=-1)¶
- write(data)¶
- class mozpack.files.ExecutableFile(path)¶
Bases:
File
File class for executable and library files on OS/2, OS/X and ELF systems. (see mozpack.executables.is_executable documentation).
- copy(dest, skip_if_older=True)¶
Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).
- class mozpack.files.ExistingFile(required)¶
Bases:
BaseFile
File class that represents a file that may exist but whose content comes from elsewhere.
This purpose of this class is to account for files that are installed via external means. It is typically only used in manifests or in registries to account for files.
When asked to copy, this class does nothing because nothing is known about the source file/data.
Instances of this class come in two flavors: required and optional. If an existing file is required, it must exist during copy() or an error is raised.
- copy(dest, skip_if_older=True)¶
Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).
- inputs()¶
Return an iterable of the input file paths that impact this output file.
- class mozpack.files.ExtractedTarFile(tar, info)¶
Bases:
GeneratedFile
File class for members of a tar archive. Contents of the underlying file are extracted immediately and stored in memory.
- property mode¶
Return the file’s unix mode, or None if it has no meaning.
- read()¶
- class mozpack.files.File(path)¶
Bases:
BaseFile
File class for plain files.
- inputs()¶
Return an iterable of the input file paths that impact this output file.
- property mode¶
Return the file’s unix mode, as returned by os.stat().st_mode.
- read()¶
Return the contents of the file.
- size()¶
Returns size of the entry.
Derived classes are highly encouraged to override this with a more optimal implementation.
- class mozpack.files.FileFinder(base, find_executables=False, ignore=(), ignore_broken_symlinks=False, find_dotfiles=False, **kargs)¶
Bases:
BaseFinder
Helper to get appropriate BaseFile instances from the file system.
- get(path)¶
Obtain a single file.
Where
find
is tailored towards matching multiple files, this method is used for retrieving a single file. Use this method when performance is critical.Returns a
BaseFile
if at most one file exists orNone
otherwise.
- class mozpack.files.FileListFinder(files)¶
Bases:
BaseFinder
Finder for a literal list of file names.
- find(pattern)¶
Yield path, BaseFile_instance pairs for all files under the base directory and its subdirectories that match the given pattern. See the mozpack.path.match documentation for a description of the handled patterns.
- class mozpack.files.GeneratedFile(content)¶
Bases:
BaseFile
File class for content with no previous existence on the filesystem.
- property content¶
- inputs()¶
Return an iterable of the input file paths that impact this output file.
- open()¶
Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.
- read()¶
- size()¶
Returns size of the entry.
Derived classes are highly encouraged to override this with a more optimal implementation.
- class mozpack.files.HardlinkFile(path)¶
Bases:
File
File class that is copied by hard linking (if available)
This is similar to the AbsoluteSymlinkFile, but with hard links. The symlink implementation requires paths to be absolute, because they are resolved at read time, which makes relative paths messy. Hard links resolve paths at link-creation time, so relative paths are fine.
- copy(dest, skip_if_older=True)¶
Copy the BaseFile content to the destination given as a string or a Dest instance. Avoids replacing existing files if the BaseFile content matches that of the destination, or in case of plain files, if the destination is newer than the original file. This latter behaviour is disabled when skip_if_older is False. Returns whether a copy was actually performed (True) or not (False).
- class mozpack.files.JarFinder(base, reader, **kargs)¶
Bases:
BaseFinder
Helper to get appropriate DeflatedFile instances from a JarReader.
- class mozpack.files.ManifestFile(base, entries=None)¶
Bases:
BaseFile
File class for a manifest file. It takes individual manifest entries (using the add() and remove() member functions), and adjusts them to be relative to the base path for the manifest, given at creation. .. rubric:: Example
There is a manifest entry “content foobar foobar/content/” relative to “foobar/chrome”. When packaging, the entry will be stored in jar:foobar/omni.ja!/chrome/chrome.manifest, which means the entry will have to be relative to “chrome” instead of “foobar/chrome”. This doesn’t really matter when serializing the entry, since this base path is not written out, but it matters when moving the entry at the same time, e.g. to jar:foobar/omni.ja!/chrome.manifest, which we don’t do currently but could in the future.
- add(entry)¶
Add the given entry to the manifest. Entries are rebased at open() time instead of add() time so that they can be more easily remove()d.
- isempty()¶
Return whether there are manifest entries to write
- open()¶
Return a file-like object allowing to read() the serialized content of the manifest.
- remove(entry)¶
Remove the given entry from the manifest.
- class mozpack.files.MercurialFile(client, rev, path)¶
Bases:
BaseFile
File class for holding data from Mercurial.
- open()¶
Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.
- read()¶
- class mozpack.files.MercurialRevisionFinder(repo, rev='.', recognize_repo_paths=False, **kwargs)¶
Bases:
BaseFinder
A finder that operates on a specific Mercurial revision.
- get(path)¶
Obtain a single file.
Where
find
is tailored towards matching multiple files, this method is used for retrieving a single file. Use this method when performance is critical.Returns a
BaseFile
if at most one file exists orNone
otherwise.
- class mozpack.files.MinifiedCommentStripped(file)¶
Bases:
BaseFile
File class for content minified by stripping comments. This wraps around a BaseFile instance, and removes lines starting with a # from its content.
- open()¶
Return a file-like object allowing to read() the minified content of the underlying file.
- class mozpack.files.MinifiedJavaScript(file, verify_command=None)¶
Bases:
BaseFile
File class for minifying JavaScript files.
- open()¶
Return a file-like object allowing to read() the content of the associated file. This is meant to be overloaded in subclasses to return a custom file-like object.
- class mozpack.files.PreprocessedFile(path, depfile_path, marker, defines, extra_depends=None, silence_missing_directive_warnings=False)¶
Bases:
BaseFile
File class for a file that is preprocessed. PreprocessedFile.copy() runs the preprocessor on the file to create the output.
- copy(dest, skip_if_older=True)¶
Invokes the preprocessor to create the destination file.
- inputs()¶
Return an iterable of the input file paths that impact this output file.
- class mozpack.files.TarFinder(base, tar, **kargs)¶
Bases:
BaseFinder
Helper to get files from a TarFile.
mozpack.macpkg module¶
- class mozpack.macpkg.CpioInfo(mode, nlink, dev, ino)¶
Bases:
tuple
- dev¶
Alias for field number 2
- ino¶
Alias for field number 3
- mode¶
Alias for field number 0
- nlink¶
Alias for field number 1
- class mozpack.macpkg.Take(fileobj, limit)¶
Bases:
object
File object wrapper that allows to read at most a certain length.
- read(length=None)¶
- mozpack.macpkg.uncpio(fileobj)¶
- mozpack.macpkg.unxar(fileobj)¶
mozpack.manifests module¶
- class mozpack.manifests.InstallManifest(path=None, fileobj=None)¶
Bases:
object
Describes actions to be used with a copier.FileCopier instance.
This class facilitates serialization and deserialization of data used to construct a copier.FileCopier and to perform copy operations.
The manifest defines source paths, destination paths, and a mechanism by which the destination file should come into existence.
Entries in the manifest correspond to the following types:
- copy – The file specified as the source path will be copied to the
destination path.
- link – The destination path will be a symlink or hardlink to the source
path. If symlinks are not supported, a copy will be performed.
- exists – The destination path is accounted for and won’t be deleted by
the FileCopier. If the destination path doesn’t exist, an error is raised.
- optional – The destination path is accounted for and won’t be deleted by
the FileCopier. No error is raised if the destination path does not exist.
- patternlink – Paths matched by the expression in the source path
will be symlinked or hardlinked to the destination directory.
- patterncopy – Similar to patternlink except files are copied, not
symlinked/hardlinked.
- preprocess – The file specified at the source path will be run through
the preprocessor, and the output will be written to the destination path.
content – The destination file will be created with the given content.
Version 1 of the manifest was the initial version. Version 2 added optional path support Version 3 added support for pattern entries. Version 4 added preprocessed file support. Version 5 added content support.
- CONTENT = 8¶
- COPY = 2¶
- CURRENT_VERSION = 5¶
- FIELD_SEPARATOR = '\x1f'¶
- LINK = 1¶
- OPTIONAL_EXISTS = 4¶
- PATTERN_COPY = 6¶
- PATTERN_LINK = 5¶
- PREPROCESS = 7¶
- REQUIRED_EXISTS = 3¶
- add_content(content, dest)¶
Add a file with the given content.
- add_copy(source, dest)¶
Add a copy to this manifest.
source will be copied to dest.
- add_entries_from(other, base='')¶
Copy data from another mozpack.copier.InstallManifest instance, adding an optional base prefix to the destination.
This allows to merge two manifests into a single manifest, or two take the tagged union of two manifests.
- add_link(source, dest)¶
Add a link to this manifest.
dest will be either a symlink or hardlink to source.
- add_optional_exists(dest)¶
Record that a destination file may exist.
This effectively prevents the listed file from being deleted. Unlike a “required exists” file, files of this type do not raise errors if the destination file does not exist.
- add_pattern_copy(base, pattern, dest)¶
Add a pattern match that results in copies.
See
add_pattern_link()
for usage.
- add_pattern_link(base, pattern, dest)¶
Add a pattern match that results in links being created.
A
FileFinder
will be created with its base set tobase
andFileFinder.find()
will be called withpattern
to discover source files. Each source file will be either symlinked or hardlinked underdest
.Filenames under
dest
are constructed by taking the path fragment afterbase
and concatenating it withdest
. e.g.<base>/foo/bar.h -> <dest>/foo/bar.h
- add_preprocess(source, dest, deps, marker='#', defines={}, silence_missing_directive_warnings=False)¶
Add a preprocessed file to this manifest.
source
will be passed through preprocessor.py, and the output will be written todest
.
- add_required_exists(dest)¶
Record that a destination file must exist.
This effectively prevents the listed file from being deleted.
- populate_registry(registry, defines_override={}, link_policy='symlink')¶
Populate a mozpack.copier.FileRegistry instance with data from us.
The caller supplied a FileRegistry instance (or at least something that conforms to its interface) and that instance is populated with data from this manifest.
Defines can be given to override the ones in the manifest for preprocessing.
The caller can set a link policy. This determines whether symlinks, hardlinks, or copies are used for LINK and PATTERN_LINK.
- write(path=None, fileobj=None, expand_pattern=False)¶
Serialize this manifest to a file or file object.
If path is specified, that file will be written to. If fileobj is specified, the serialized content will be written to that file object.
It is an error if both are specified.
- exception mozpack.manifests.UnreadableInstallManifest¶
Bases:
Exception
Raised when an invalid install manifest is parsed.
mozpack.mozjar module¶
- class mozpack.mozjar.Deflater(compress=True, compress_level=9)¶
Bases:
object
File-like interface to zlib compression. The data is actually not compressed unless the compressed form is smaller than the uncompressed data.
- close()¶
Close the Deflater.
- property compressed¶
Return whether the data should be compressed.
- property compressed_data¶
Return the compressed data, if the data should be compressed (real compressed size smaller than the uncompressed size), or the uncompressed data otherwise.
- property compressed_size¶
Return the compressed size of the data written to the Deflater. If the Deflater is set not to compress, the uncompressed size is returned. Otherwise, if the data should not be compressed (the real compressed size is bigger than the uncompressed size), return the uncompressed size.
- property uncompressed_size¶
Return the size of the data written to the Deflater.
- write(data)¶
Append a buffer to the Deflater.
- class mozpack.mozjar.JarCdirEnd(data=None)¶
Bases:
JarStruct
End of central directory record.
- MAGIC = 101010256¶
- STRUCT = {'cdir_disk': 'uint16', 'cdir_entries': 'uint16', 'cdir_offset': 'uint32', 'cdir_size': 'uint32', 'comment': 'comment_size', 'comment_size': 'uint16', 'disk_entries': 'uint16', 'disk_num': 'uint16'}¶
- class mozpack.mozjar.JarCdirEntry(data=None)¶
Bases:
JarStruct
Central directory file header
- MAGIC = 33639248¶
- STRUCT = {'compressed_size': 'uint32', 'compression': 'uint16', 'crc32': 'uint32', 'creator_version': 'uint16', 'disknum': 'uint16', 'external_attr': 'uint32', 'extrafield': 'extrafield_size', 'extrafield_size': 'uint16', 'filecomment': 'filecomment_size', 'filecomment_size': 'uint16', 'filename': 'filename_size', 'filename_size': 'uint16', 'general_flag': 'uint16', 'internal_attr': 'uint16', 'lastmod_date': 'uint16', 'lastmod_time': 'uint16', 'min_version': 'uint16', 'offset': 'uint32', 'uncompressed_size': 'uint32'}¶
- class mozpack.mozjar.JarFileReader(header, data)¶
Bases:
object
File-like class for use by JarReader to give access to individual files within a Jar archive.
- close()¶
Free the uncompressed data buffer.
- property closed¶
- property compressed_data¶
Return the raw compressed data.
- read(length=-1)¶
Read some amount of uncompressed data.
- readable()¶
- readinto(b)¶
Read bytes into a pre-allocated, writable bytes-like object b and return the number of bytes read.
- readlines()¶
Return a list containing all the lines of data in the uncompressed data.
- seek(pos, whence=0)¶
Change the current position in the uncompressed data. Subsequent reads will start from there.
- property uncompressed_data¶
Return the uncompressed data.
- class mozpack.mozjar.JarLocalFileHeader(data=None)¶
Bases:
JarStruct
Local file header
- MAGIC = 67324752¶
- STRUCT = {'compressed_size': 'uint32', 'compression': 'uint16', 'crc32': 'uint32', 'extra_field': 'extra_field_size', 'extra_field_size': 'uint16', 'filename': 'filename_size', 'filename_size': 'uint16', 'general_flag': 'uint16', 'lastmod_date': 'uint16', 'lastmod_time': 'uint16', 'min_version': 'uint16', 'uncompressed_size': 'uint32'}¶
- class mozpack.mozjar.JarLog(file=None, fileobj=None)¶
Bases:
dict
Helper to read the file Gecko generates when setting MOZ_JAR_LOG_FILE. The jar log is then available as a dict with the jar path as key, and the corresponding access log as a list value. Only the first access to a given member of a jar is stored.
- class mozpack.mozjar.JarReader(file=None, fileobj=None, data=None)¶
Bases:
object
Class with methods to read Jar files. Can open standard jar files as well as Mozilla jar files (see further details in the JarWriter documentation).
- close()¶
Free some resources associated with the Jar.
- property compression¶
- property entries¶
Return an ordered dict of central directory entries, indexed by filename, in the order they appear in the Jar archive central directory. Directory entries are skipped.
- property is_optimized¶
Return whether the jar archive is optimized.
- property last_preloaded¶
Return the name of the last file that is set to be preloaded. See JarWriter documentation for more details on preloading.
- exception mozpack.mozjar.JarReaderError¶
Bases:
Exception
Error type for Jar reader errors.
- class mozpack.mozjar.JarStruct(data=None)¶
Bases:
object
Helper used to define ZIP archive raw data structures. Data structures handled by this helper all start with a magic number, defined in subclasses MAGIC field as a 32-bits unsigned integer, followed by data structured as described in subclasses STRUCT field.
The STRUCT field contains a list of (name, type) pairs where name is a field name, and the type can be one of ‘uint32’, ‘uint16’ or one of the field names. In the latter case, the field is considered to be a string buffer with a length given in that field. For example,
STRUCT = [ ('version', 'uint32'), ('filename_size', 'uint16'), ('filename', 'filename_size') ]
describes a structure with a ‘version’ 32-bits unsigned integer field, followed by a ‘filename_size’ 16-bits unsigned integer field, followed by a filename_size-long string buffer ‘filename’.
Fields that are used as other fields size are not stored in objects. In the above example, an instance of such subclass would only have two attributes:
obj[‘version’]
obj[‘filename’]
filename_size would be obtained with len(obj[‘filename’]).
JarStruct subclasses instances can be either initialized from existing data (deserialized), or with empty fields.
- TYPE_MAPPING = {'uint16': (b'H', 2), 'uint32': (b'I', 4)}¶
- static get_data(type, data)¶
Deserialize a single field of given type (must be one of JarStruct.TYPE_MAPPING) at the given offset in the given data.
- serialize()¶
Serialize the data structure according to the data structure definition from self.STRUCT.
- property size¶
Return the size of the data structure, given the current values of all variable length fields.
- class mozpack.mozjar.JarWriter(file=None, fileobj=None, compress=True, compress_level=9)¶
Bases:
object
Class with methods to write Jar files. Can write more-or-less standard jar archives as well as jar archives optimized for Gecko. See the documentation for the close() member function for a description of both layouts.
- add(name, data, compress=None, mode=None, skip_duplicates=False)¶
Add a new member to the jar archive, with the given name and the given data. The compress option indicates how the given data should be compressed (one of JAR_STORED or JAR_DEFLATE), or compressed according to the default defined when creating the JarWriter (None). True and False are allowed values for backwards compatibility, mapping, respectively, to JAR_DEFLATE and JAR_STORED. When the data should be compressed, it is only really compressed if the compressed size is smaller than the uncompressed size. The mode option gives the unix permissions that should be stored for the jar entry, which defaults to 0o100644 (regular file, u+rw, g+r, o+r) if not specified. If a duplicated member is found skip_duplicates will prevent raising an exception if set to True. The given data may be a buffer, a file-like instance, a Deflater or a JarFileReader instance. The latter two allow to avoid uncompressing data to recompress it.
- finish()¶
Flush and close the Jar archive.
- Standard jar archives are laid out like the following:
Local file header 1
File data 1
Local file header 2
File data 2
(…)
Central directory entry pointing at Local file header 1
Central directory entry pointing at Local file header 2
(…)
End of central directory, pointing at first central directory entry.
- Jar archives optimized for Gecko are laid out like the following:
32-bits unsigned integer giving the amount of data to preload.
Central directory entry pointing at Local file header 1
Central directory entry pointing at Local file header 2
(…)
End of central directory, pointing at first central directory entry.
Local file header 1
File data 1
Local file header 2
File data 2
(…)
End of central directory, pointing at first central directory entry.
The duplication of the End of central directory is to accomodate some Zip reading tools that want an end of central directory structure to follow the central directory entries.
- preload(files)¶
Set which members of the jar archive should be preloaded when opening the archive in Gecko. This reorders the members according to the order of given list.
- exception mozpack.mozjar.JarWriterError¶
Bases:
Exception
Error type for Jar writer errors.
mozpack.path module¶
Like os.path
, with a reduced set of functions, and with normalized path
separators (always use forward slashes).
Also contains a few additional utilities not found in os.path
.
- mozpack.path.abspath(path)¶
- mozpack.path.basedir(path, bases)¶
Given a list of directories (bases), return which one contains the given path. If several matches are found, the deepest base directory is returned.
basedir('foo/bar/baz', ['foo', 'baz', 'foo/bar'])
returns'foo/bar'
(‘foo’ and ‘foo/bar’ both match, but ‘foo/bar’ is the deepest match)
- mozpack.path.basename(path)¶
- mozpack.path.cargo_workaround(path)¶
- mozpack.path.commonprefix(paths)¶
- mozpack.path.dirname(path)¶
- mozpack.path.join(*paths)¶
- mozpack.path.match(path, pattern)¶
Return whether the given path matches the given pattern. An asterisk can be used to match any string, including the null string, in one part of the path:
foo
matches*
,f*
orfo*o
However, an asterisk matching a subdirectory may not match the null string:
foo/bar
does not matchfoo/*/bar
If the pattern matches one of the ancestor directories of the path, the patch is considered matching:
foo/bar
matchesfoo
Two adjacent asterisks can be used to match files and zero or more directories and subdirectories.
foo/bar
matchesfoo/**/bar
, or**/bar
- mozpack.path.normpath(path)¶
- mozpack.path.normsep(path)¶
Normalize path separators, by using forward slashes instead of whatever
os.sep
is.
- mozpack.path.readlink(path)¶
- mozpack.path.realpath(path)¶
- mozpack.path.rebase(oldbase, base, relativepath)¶
Return relativepath relative to base instead of oldbase.
- mozpack.path.relpath(path, start)¶
- mozpack.path.split(path)¶
Return the normalized path as a list of its components.
split('foo/bar/baz')
returns['foo', 'bar', 'baz']
- mozpack.path.splitext(path)¶
mozpack.pkg module¶
- mozpack.pkg.create_bom(bom_path: Path, root_path: Path, mkbom_tool: Path)¶
Creates a Bill Of Materials file at <bom_path> based on <root_path>
- Parameters:
bom_path – Path, destination Path for the BOM file
root_path – Path, root directory Path
mkbom_tool – Path, mkbom tool Path
- mozpack.pkg.create_payload(destination: Path, root_path: Path, cpio_tool: str)¶
Creates a payload at <destination> based on <root_path>
- Parameters:
destination – Path, the destination Path
root_path – Path, the root directory Path
cpio_tool – str,
- mozpack.pkg.create_pkg(source_app: Path, output_pkg: Path, mkbom_tool: Path, xar_tool: Path, cpio_tool: Path)¶
Create a mac PKG installer from <source_app> to <output_pkg>
- Parameters:
source_app – Path, source .app file/directory Path
output_pkg – Path, destination .pkg file
mkbom_tool – Path, mkbom tool Path
xar_tool – Path, xar tool Path
cpio – Path, cpio tool Path
- mozpack.pkg.get_app_info_plist(app_path: Path) dict ¶
Retrieve most information from Info.plist file of an app. The Info.plist file should be located in ?.app/Contents/Info.plist
Note: Ignores properties that are not <string> type
- Parameters:
app_path – Path, the .app file/directory path
- Returns:
dict, the dictionary of properties found in Info.plist
- mozpack.pkg.get_apple_template(name: str) Template ¶
- Given <name>, open file at <TEMPLATE_DIRECTORY>/<name>, read contents and
return as a Template
- Parameters:
name – str, Filename for the template
- Returns:
Template, loaded from file
- mozpack.pkg.get_relative_glob_list(source: Path, glob: str) List[str] ¶
Given a source path, return a list of relative path based on glob
- Parameters:
source – Path, source directory Path
glob – str, unix style glob
- Returns:
list[str], paths found in source directory
- mozpack.pkg.save_text_file(content: str, destination: Path)¶
Saves a text file to <destination> with provided <content> Note: Overwrites contents
- Parameters:
content – str, The desired contents of the file
destination – Path, The file path
- mozpack.pkg.xar_package_folder(source_path: Path, destination: Path, xar_tool: Path)¶
Create a pkg from <source_path> to <destination> The command is issued with <source_path> as cwd
- Parameters:
source_path – Path, source absolute Path
destination – Path, destination absolute Path
xar_tool – Path, xar tool Path
mozpack.unify module¶
- class mozpack.unify.UnifiedBuildFinder(finder1, finder2, **kargs)¶
Bases:
UnifiedFinder
Specialized UnifiedFinder for Mozilla applications packaging. It allows
*.manifest
files to differ in their order, and unifiesbuildconfig.html
files by merging their content.- unify_file(path, file1, file2)¶
Unify files taking Mozilla application special cases into account. Otherwise defer to UnifiedFinder.unify_file.
- class mozpack.unify.UnifiedExecutableFile(executable1, executable2)¶
Bases:
BaseFile
File class for executable and library files that to be unified with ‘lipo’.
- copy(dest, skip_if_older=True)¶
Create a fat executable from the two Mach-O executable given when creating the instance. skip_if_older is ignored.
- class mozpack.unify.UnifiedFinder(finder1, finder2, sorted=[], **kargs)¶
Bases:
BaseFinder
Helper to get unified BaseFile instances from two distinct trees on the file system.
- unify_file(path, file1, file2)¶
Given two BaseFiles and the path they were found at, return a unified version of the files. If the files match, the first BaseFile may be returned. If the files don’t match or one of them is None, the method returns None. Subclasses may decide to unify by using one of the files in that case.
- mozpack.unify.may_unify_binary(file)¶
Return whether the given BaseFile instance is an ExecutableFile that may be unified. Only non-fat Mach-O binaries are to be unified.