|
HTML rendering created 2024-06-26 by Michael Kerrisk, author of The Linux Programming Interface. For details of in-depth Linux/UNIX system programming training courses that I teach, look here. Hosting by jambit GmbH. |
|
|
PROLOG | NAME | SYNOPSIS | DESCRIPTION | OPTIONS | OPERANDS | STDIN | INPUT FILES | ENVIRONMENT VARIABLES | ASYNCHRONOUS EVENTS | STDOUT | STDERR | OUTPUT FILES | EXTENDED DESCRIPTION | EXIT STATUS | CONSEQUENCES OF ERRORS | APPLICATION USAGE | EXAMPLES | RATIONALE | FUTURE DIRECTIONS | SEE ALSO | COPYRIGHT |
|
|
|
PAX(1P) POSIX Programmer's Manual PAX(1P)
This manual page is part of the POSIX Programmer's Manual. The
Linux implementation of this interface may differ (consult the
corresponding Linux manual page for details of Linux behavior),
or the interface may not be implemented on Linux.
pax — portable archive interchange
pax [-dv] [-c|-n] [-H|-L] [-o options] [-f archive] [-s replstr]...
[pattern...]
pax -r[-c|-n] [-dikuv] [-H|-L] [-f archive] [-o options]... [-p string]...
[-s replstr]... [pattern...]
pax -w [-dituvX] [-H|-L] [-b blocksize] [[-a] [-f archive]] [-o options]...
[-s replstr]... [-x format] [file...]
pax -r -w [-diklntuvX] [-H|-L] [-o options]... [-p string]...
[-s replstr]... [file...] directory
The pax utility shall read, write, and write lists of the members
of archive files and copy directory hierarchies. A variety of
archive formats shall be supported; see the -x format option.
The action to be taken depends on the presence of the -r and -w
options. The four combinations of -r and -w are referred to as
the four modes of operation: list, read, write, and copy modes,
corresponding respectively to the four forms shown in the
SYNOPSIS section.
list In list mode (when neither -r nor -w are specified),
pax shall write the names of the members of the archive
file read from the standard input, with pathnames
matching the specified patterns, to standard output. If
a named file is of type directory, the file hierarchy
rooted at that file shall be listed as well.
read In read mode (when -r is specified, but -w is not), pax
shall extract the members of the archive file read from
the standard input, with pathnames matching the
specified patterns. If an extracted file is of type
directory, the file hierarchy rooted at that file shall
be extracted as well. The extracted files shall be
created performing pathname resolution with the
directory in which pax was invoked as the current
working directory.
If an attempt is made to extract a directory when the
directory already exists, this shall not be considered
an error. If an attempt is made to extract a FIFO when
the FIFO already exists, this shall not be considered
an error.
The ownership, access, and modification times, and file
mode of the restored files are discussed under the -p
option.
write In write mode (when -w is specified, but -r is not),
pax shall write the contents of the file operands to
the standard output in an archive format. If no file
operands are specified, a list of files to copy, one
per line, shall be read from the standard input and
each entry in this list shall be processed as if it had
been a file operand on the command line. A file of type
directory shall include all of the files in the file
hierarchy rooted at the file.
copy In copy mode (when both -r and -w are specified), pax
shall copy the file operands to the destination
directory.
If no file operands are specified, a list of files to
copy, one per line, shall be read from the standard
input. A file of type directory shall include all of
the files in the file hierarchy rooted at the file.
The effect of the copy shall be as if the copied files
were written to a pax format archive file and then
subsequently extracted, except that copying of sockets
may be supported even if archiving them in write mode
is not supported, and that there may be hard links
between the original and the copied files. If the
destination directory is a subdirectory of one of the
files to be copied, the results are unspecified. If the
destination directory is a file of a type not defined
by the System Interfaces volume of POSIX.1‐2017, the
results are implementation-defined; otherwise, it shall
be an error for the file named by the directory operand
not to exist, not be writable by the user, or not be a
file of type directory.
In read or copy modes, if intermediate directories are necessary
to extract an archive member, pax shall perform actions
equivalent to the mkdir() function defined in the System
Interfaces volume of POSIX.1‐2017, called with the following
arguments:
* The intermediate directory used as the path argument
* The value of the bitwise-inclusive OR of S_IRWXU, S_IRWXG,
and S_IRWXO as the mode argument
If any specified pattern or file operands are not matched by at
least one file or archive member, pax shall write a diagnostic
message to standard error for each one that did not match and
exit with a non-zero exit status.
The archive formats described in the EXTENDED DESCRIPTION section
shall be automatically detected on input. The default output
archive format shall be implementation-defined.
A single archive can span multiple files. The pax utility shall
determine, in an implementation-defined manner, what file to read
or write as the next file.
If the selected archive format supports the specification of
linked files, it shall be an error if these files cannot be
linked when the archive is extracted. For archive formats that do
not store file contents with each name that causes a hard link,
if the file that contains the data is not extracted during this
pax session, either the data shall be restored from the original
file, or a diagnostic message shall be displayed with the name of
a file that can be used to extract the data. In traversing
directories, pax shall detect infinite loops; that is, entering a
previously visited directory that is an ancestor of the last file
visited. When it detects an infinite loop, pax shall write a
diagnostic message to standard error and shall terminate.
The pax utility shall conform to the Base Definitions volume of
POSIX.1‐2017, Section 12.2, Utility Syntax Guidelines, except
that the order of presentation of the -o, -p, and -s options is
significant.
The following options shall be supported:
-r Read an archive file from standard input.
-w Write files to the standard output in the specified
archive format.
-a Append files to the end of the archive. It is
implementation-defined which devices on the system
support appending. Additional file formats unspecified
by this volume of POSIX.1‐2017 may impose restrictions
on appending.
-b blocksize
Block the output at a positive decimal integer number
of bytes per write to the archive file. Devices and
archive formats may impose restrictions on blocking.
Blocking shall be automatically determined on input.
Conforming applications shall not specify a blocksize
value larger than 32256. Default blocking when creating
archives depends on the archive format. (See the -x
option below.)
-c Match all file or archive members except those
specified by the pattern or file operands.
-d Cause files of type directory being copied or archived
or archive members of type directory being extracted or
listed to match only the file or archive member itself
and not the file hierarchy rooted at the file.
-f archive
Specify the pathname of the input or output archive,
overriding the default standard input (in list or read
modes) or standard output (write mode).
-H If a symbolic link referencing a file of type directory
is specified on the command line, pax shall archive the
file hierarchy rooted in the file referenced by the
link, using the name of the link as the root of the
file hierarchy. Otherwise, if a symbolic link
referencing a file of any other file type which pax can
normally archive is specified on the command line, then
pax shall archive the file referenced by the link,
using the name of the link. The default behavior, when
neither -H or -L are specified, shall be to archive the
symbolic link itself.
-i Interactively rename files or archive members. For each
archive member matching a pattern operand or file
matching a file operand, a prompt shall be written to
the file /dev/tty. The prompt shall contain the name
of the file or archive member, but the format is
otherwise unspecified. A line shall then be read from
/dev/tty. If this line is blank, the file or archive
member shall be skipped. If this line consists of a
single period, the file or archive member shall be
processed with no modification to its name. Otherwise,
its name shall be replaced with the contents of the
line. The pax utility shall immediately exit with a
non-zero exit status if end-of-file is encountered when
reading a response or if /dev/tty cannot be opened for
reading and writing.
The results of extracting a hard link to a file that
has been renamed during extraction are unspecified.
-k Prevent the overwriting of existing files.
-l (The letter ell.) In copy mode, hard links shall be
made between the source and destination file
hierarchies whenever possible. If specified in
conjunction with -H or -L, when a symbolic link is
encountered, the hard link created in the destination
file hierarchy shall be to the file referenced by the
symbolic link. If specified when neither -H nor -L is
specified, when a symbolic link is encountered, the
implementation shall create a hard link to the symbolic
link in the source file hierarchy or copy the symbolic
link to the destination.
-L If a symbolic link referencing a file of type directory
is specified on the command line or encountered during
the traversal of a file hierarchy, pax shall archive
the file hierarchy rooted in the file referenced by the
link, using the name of the link as the root of the
file hierarchy. Otherwise, if a symbolic link
referencing a file of any other file type which pax can
normally archive is specified on the command line or
encountered during the traversal of a file hierarchy,
pax shall archive the file referenced by the link,
using the name of the link. The default behavior, when
neither -H or -L are specified, shall be to archive the
symbolic link itself.
-n Select the first archive member that matches each
pattern operand. No more than one archive member shall
be matched for each pattern (although members of type
directory shall still match the file hierarchy rooted
at that file).
-o options
Provide information to the implementation to modify the
algorithm for extracting or writing files. The value of
options shall consist of one or more <comma>-separated
keywords of the form:
keyword[[:]=value][,keyword[[:]=value], ...]
Some keywords apply only to certain file formats, as
indicated with each description. Use of keywords that
are inapplicable to the file format being processed
produces undefined results.
Keywords in the options argument shall be a string that
would be a valid portable filename as described in the
Base Definitions volume of POSIX.1‐2017, Section 3.282,
Portable Filename Character Set.
Note: Keywords are not expected to be filenames,
merely to follow the same character composition
rules as portable filenames.
Keywords can be preceded with white space. The value
field shall consist of zero or more characters; within
value, the application shall precede any literal
<comma> with a <backslash>, which shall be ignored, but
preserves the <comma> as part of value. A <comma> as
the final character, or a <comma> followed solely by
white space as the final characters, in options shall
be ignored. Multiple -o options can be specified; if
keywords given to these multiple -o options conflict,
the keywords and values appearing later in command line
sequence shall take precedence and the earlier shall be
silently ignored. The following keyword values of
options shall be supported for the file formats as
indicated:
delete=pattern
(Applicable only to the -x pax format.) When used
in write or copy mode, pax shall omit from
extended header records that it produces any
keywords matching the string pattern. When used
in read or list mode, pax shall ignore any
keywords matching the string pattern in the
extended header records. In both cases, matching
shall be performed using the pattern matching
notation described in Section 2.13.1, Patterns
Matching a Single Character and Section 2.13.2,
Patterns Matching Multiple Characters. For
example:
-o delete=security.*
would suppress security-related information. See
pax Extended Header for extended header record
keyword usage.
When multiple -odelete=pattern options are
specified, the patterns shall be additive; all
keywords matching the specified string patterns
shall be omitted from extended header records
that pax produces.
exthdr.name=string
(Applicable only to the -x pax format.) This
keyword allows user control over the name that is
written into the ustar header blocks for the
extended header produced under the circumstances
described in pax Header Block. The name shall be
the contents of string, after the following
character substitutions have been made:
┌───────────┬────────────────────────────────────────┐
│ string │ │
│ Includes: │ Replaced by: │
├───────────┼────────────────────────────────────────┤
│ %d │ The directory name of the file, │
│ │ equivalent to the result of the │
│ │ dirname utility on the translated │
│ │ pathname. │
│ %f │ The filename of the file, equivalent │
│ │ to the result of the basename utility │
│ │ on the translated pathname. │
│ %p │ The process ID of the pax process. │
│ %% │ A '%' character. │
└───────────┴────────────────────────────────────────┘
Any other '%' characters in string produce
undefined results.
If no -o exthdr.name=string is specified, pax
shall use the following default value:
%d/PaxHeaders.%p/%f
globexthdr.name=string
(Applicable only to the -x pax format.) When used
in write or copy mode with the appropriate
options, pax shall create global extended header
records with ustar header blocks that will be
treated as regular files by previous versions of
pax. This keyword allows user control over the
name that is written into the ustar header blocks
for global extended header records. The name
shall be the contents of string, after the
following character substitutions have been made:
┌───────────┬────────────────────────────────────────┐
│ string │ │
│ Includes: │ Replaced by: │
├───────────┼────────────────────────────────────────┤
│ %n │ An integer that represents the │
│ │ sequence number of the global extended │
│ │ header record in the archive, starting │
│ │ at 1. │
│ %p │ The process ID of the pax process. │
│ %% │ A '%' character. │
└───────────┴────────────────────────────────────────┘
Any other '%' characters in string produce
undefined results.
If no -o globexthdr.name=string is specified, pax
shall use the following default value:
$TMPDIR/GlobalHead.%p.%n
where $TMPDIR represents the value of the TMPDIR
environment variable. If TMPDIR is not set, pax
shall use /tmp.
invalid=action
(Applicable only to the -x pax format.) This
keyword allows user control over the action pax
takes upon encountering values in an extended
header record that, in read or copy mode, are
invalid in the destination hierarchy or, in list
mode, cannot be written in the codeset and
current locale of the implementation. The
following are invalid values that shall be
recognized by pax:
-- In read or copy mode, a filename or link name
that contains character encodings invalid in
the destination hierarchy. (For example, the
name may contain embedded NULs.)
-- In read or copy mode, a filename or link name
that is longer than the maximum allowed in
the destination hierarchy (for either a
pathname component or the entire pathname).
-- In list mode, any character string value
(filename, link name, user name, and so on)
that cannot be written in the codeset and
current locale of the implementation.
The following mutually-exclusive values of the
action argument are supported:
binary In write mode, pax shall generate a
hdrcharset=BINARY extended header
record for each file with a filename,
link name, group name, owner name, or
any other field in an extended header
record that cannot be translated to the
UTF‐8 codeset, allowing the archive to
contain the files with unencoded
extended header record values. In read
or copy mode, pax shall use the values
specified in the header without
translation, regardless of whether this
may overwrite an existing file with a
valid name. In list mode, pax shall
behave identically to the bypass
action.
bypass In read or copy mode, pax shall bypass
the file, causing no change to the
destination hierarchy. In list mode,
pax shall write all requested valid
values for the file, but its method for
writing invalid values is unspecified.
rename In read or copy mode, pax shall act as
if the -i option were in effect for
each file with invalid filename or link
name values, allowing the user to
provide a replacement name
interactively. In list mode, pax shall
behave identically to the bypass
action.
UTF‐8 When used in read, copy, or list mode
and a filename, link name, owner name,
or any other field in an extended
header record cannot be translated from
the pax UTF‐8 codeset format to the
codeset and current locale of the
implementation, pax shall use the
actual UTF‐8 encoding for the name. If
a hdrcharset extended header record is
in effect for this file, the character
set specified by that record shall be
used instead of UTF‐8. If a
hdrcharset=BINARY extended header
record is in effect for this file, no
translation shall be performed.
write In read or copy mode, pax shall write
the file, translating the name,
regardless of whether this may
overwrite an existing file with a valid
name. In list mode, pax shall behave
identically to the bypass action.
If no -o invalid=option is specified, pax shall
act as if -oinvalid=bypass were specified. Any
overwriting of existing files that may be allowed
by the -oinvalid= actions shall be subject to
permission (-p) and modification time (-u)
restrictions, and shall be suppressed if the -k
option is also specified.
linkdata
(Applicable only to the -x pax format.) In write
mode, pax shall write the contents of a file to
the archive even when that file is merely a hard
link to a file whose contents have already been
written to the archive.
listopt=format
This keyword specifies the output format of the
table of contents produced when the -v option is
specified in list mode. See List Mode Format
Specifications. To avoid ambiguity, the
listopt=format shall be the only or final
keyword=value pair in a -o option-argument; all
characters in the remainder of the option-
argument shall be considered part of the format
string. When multiple -olistopt=format options
are specified, the format strings shall be
considered a single, concatenated string,
evaluated in command line order.
times
(Applicable only to the -x pax format.) When used
in write or copy mode, pax shall include atime
and mtime extended header records for each file.
See pax Extended Header File Times.
In addition to these keywords, if the -x pax format is
specified, any of the keywords and values defined in
pax Extended Header, including implementation
extensions, can be used in -o option-arguments, in
either of two modes:
keyword=value
When used in write or copy mode, these
keyword/value pairs shall be included at the
beginning of the archive as typeflag g global
extended header records. When used in read or
list mode, these keyword/value pairs shall act as
if they had been at the beginning of the archive
as typeflag g global extended header records.
keyword:=value
When used in write or copy mode, these
keyword/value pairs shall be included as records
at the beginning of a typeflag x extended header
for each file. (This shall be equivalent to the
<equals-sign> form except that it creates no
typeflag g global extended header records.) When
used in read or list mode, these keyword/value
pairs shall act as if they were included as
records at the end of each extended header; thus,
they shall override any global or file-specific
extended header record keywords of the same
names. For example, in the command:
pax -r -o "
gname:=mygroup,
" <archive
the group name will be forced to a new value for
all files read from the archive.
The precedence of -o keywords over various fields in
the archive is described in pax Extended Header Keyword
Precedence. If the -o delete=pattern, -o
keyword=value, or -o keyword:=value options are used to
override or remove any extended header data needed to
find files in an archive (e.g., -o delete=size for a
file whose size cannot be represented in a ustar header
or -o size=100 for a file whose size is not 100 bytes),
the behavior is undefined.
-p string Specify one or more file characteristic options
(privileges). The string option-argument shall be a
string specifying file characteristics to be retained
or discarded on extraction. The string shall consist of
the specification characters a, e, m, o, and p. Other
implementation-defined characters can be included.
Multiple characteristics can be concatenated within the
same string and multiple -p options can be specified.
The meaning of the specification characters are as
follows:
a Do not preserve file access times.
e Preserve the user ID, group ID, file mode bits
(see the Base Definitions volume of POSIX.1‐2017,
Section 3.169, File Mode Bits), access time,
modification time, and any other implementation-
defined file characteristics.
m Do not preserve file modification times.
o Preserve the user ID and group ID.
p Preserve the file mode bits. Other
implementation-defined file mode attributes may
be preserved.
In the preceding list, ``preserve'' indicates that an
attribute stored in the archive shall be given to the
extracted file, subject to the permissions of the
invoking process. The access and modification times of
the file shall be preserved unless otherwise specified
with the -p option or not stored in the archive. All
attributes that are not preserved shall be determined
as part of the normal file creation action (see Section
1.1.1.4, File Read, Write, and Creation).
If neither the e nor the o specification character is
specified, or the user ID and group ID are not
preserved for any reason, pax shall not set the S_ISUID
and S_ISGID bits of the file mode.
If the preservation of any of these items fails for any
reason, pax shall write a diagnostic message to
standard error. Failure to preserve these items shall
affect the final exit status, but shall not cause the
extracted file to be deleted.
If file characteristic letters in any of the string
option-arguments are duplicated or conflict with each
other, the ones given last shall take precedence. For
example, if -p eme is specified, file modification
times are preserved.
-s replstr
Modify file or archive member names named by pattern or
file operands according to the substitution expression
replstr, using the syntax of the ed utility. The
concepts of ``address'' and ``line'' are meaningless in
the context of the pax utility, and shall not be
supplied. The format shall be:
-s /old/new/[gp]
where as in ed, old is a basic regular expression and
new can contain an <ampersand>, '\n' (where n is a
digit) back-references, or subexpression matching. The
old string shall also be permitted to contain <newline>
characters.
Any non-null character can be used as a delimiter ('/'
shown here). Multiple -s expressions can be specified;
the expressions shall be applied in the order
specified, terminating with the first successful
substitution. The optional trailing 'g' is as defined
in the ed utility. The optional trailing 'p' shall
cause successful substitutions to be written to
standard error. File or archive member names that
substitute to the empty string shall be ignored when
reading and writing archives.
-t When reading files from the file system, and if the
user has the permissions required by utime() to do so,
set the access time of each file read to the access
time that it had before being read by pax.
-u Ignore files that are older (having a less recent file
modification time) than a pre-existing file or archive
member with the same name. In read mode, an archive
member with the same name as a file in the file system
shall be extracted if the archive member is newer than
the file. In write mode, an archive file member with
the same name as a file in the file system shall be
superseded if the file is newer than the archive
member. If -a is also specified, this is accomplished
by appending to the archive; otherwise, it is
unspecified whether this is accomplished by actual
replacement in the archive or by appending to the
archive. In copy mode, the file in the destination
hierarchy shall be replaced by the file in the source
hierarchy or by a link to the file in the source
hierarchy if the file in the source hierarchy is newer.
-v In list mode, produce a verbose table of contents (see
the STDOUT section). Otherwise, write archive member
pathnames to standard error (see the STDERR section).
-x format Specify the output archive format. The pax utility
shall support the following formats:
cpio The cpio interchange format; see the EXTENDED
DESCRIPTION section. The default blocksize
for this format for character special archive
files shall be 5120. Implementations shall
support all blocksize values less than or
equal to 32256 that are multiples of 512.
pax The pax interchange format; see the EXTENDED
DESCRIPTION section. The default blocksize
for this format for character special archive
files shall be 5120. Implementations shall
support all blocksize values less than or
equal to 32256 that are multiples of 512.
ustar The tar interchange format; see the EXTENDED
DESCRIPTION section. The default blocksize
for this format for character special archive
files shall be 10240. Implementations shall
support all blocksize values less than or
equal to 32256 that are multiples of 512.
Implementation-defined formats shall specify a default
block size as well as any other block sizes supported
for character special archive files.
Any attempt to append to an archive file in a format
different from the existing archive format shall cause
pax to exit immediately with a non-zero exit status.
-X When traversing the file hierarchy specified by a
pathname, pax shall not descend into directories that
have a different device ID (st_dev; see the System
Interfaces volume of POSIX.1‐2017, stat()).
Specifying more than one of the mutually-exclusive options -H and
-L shall not be considered an error and the last option specified
shall determine the behavior of the utility.
The options that operate on the names of files or archive members
(-c, -i, -n, -s, -u, and -v) shall interact as follows. In read
mode, the archive members shall be selected based on the user-
specified pattern operands as modified by the -c, -n, and -u
options. Then, any -s and -i options shall modify, in that order,
the names of the selected files. The -v option shall write names
resulting from these modifications.
In write mode, the files shall be selected based on the user-
specified pathnames as modified by the -n and -u options. Then,
any -s and -i options shall modify, in that order, the names of
these selected files. The -v option shall write names resulting
from these modifications.
If both the -u and -n options are specified, pax shall not
consider a file selected unless it is newer than the file to
which it is compared.
List Mode Format Specifications
In list mode with the -o listopt=format option, the format
argument shall be applied for each selected file. The pax utility
shall append a <newline> to the listopt output for each selected
file. The format argument shall be used as the format string
described in the Base Definitions volume of POSIX.1‐2017, Chapter
5, File Format Notation, with the exceptions 1. through 6.
defined in the EXTENDED DESCRIPTION section of printf, plus the
following exceptions:
7. The sequence (keyword) can occur before a format conversion
specifier. The conversion argument is defined by the value
of keyword. The implementation shall support the following
keywords:
-- Any of the Field Name entries in Table 4-14, ustar
Header Block and Table 4-16, Octet-Oriented cpio
Archive Entry. The implementation may support the cpio
keywords without the leading c_ in addition to the form
required by Table 4-16, Octet-Oriented cpio Archive
Entry.
-- Any keyword defined for the extended header in pax
Extended Header.
-- Any keyword provided as an implementation-defined
extension within the extended header defined in pax
Extended Header.
For example, the sequence "%(charset)s" is the string value
of the name of the character set in the extended header.
The result of the keyword conversion argument shall be the
value from the applicable header field or extended header,
without any trailing NULs.
All keyword values used as conversion arguments shall be
translated from the UTF‐8 encoding (or alternative encoding
specified by any hdrcharset extended header record) to the
character set appropriate for the local file system, user
database, and so on, as applicable.
8. An additional conversion specifier character, T, shall be
used to specify time formats. The T conversion specifier
character can be preceded by the sequence
(keyword=subformat), where subformat is a date format as
defined by date operands. The default keyword shall be
mtime and the default subformat shall be:
%b %e %H:%M %Y
9. An additional conversion specifier character, M, shall be
used to specify the file mode string as defined in ls
Standard Output. If (keyword) is omitted, the mode keyword
shall be used. For example, %.1M writes the single
character corresponding to the <entry type> field of the ls
-l command.
10. An additional conversion specifier character, D, shall be
used to specify the device for block or special files, if
applicable, in an implementation-defined format. If not
applicable, and (keyword) is specified, then this
conversion shall be equivalent to %(keyword)u. If not
applicable, and (keyword) is omitted, then this conversion
shall be equivalent to <space>.
11. An additional conversion specifier character, F, shall be
used to specify a pathname. The F conversion character can
be preceded by a sequence of <comma>-separated keywords:
(keyword[,keyword] ... )
The values for all the keywords that are non-null shall be
concatenated together, each separated by a '/'. The
default shall be (path) if the keyword path is defined;
otherwise, the default shall be (prefix,name).
12. An additional conversion specifier character, L, shall be
used to specify a symbolic link expansion. If the current
file is a symbolic link, then %L shall expand to:
"%s -> %s", <value of keyword>, <contents of link>
Otherwise, the %L conversion specification shall be the
equivalent of %F.
The following operands shall be supported:
directory The destination directory pathname for copy mode.
file A pathname of a file to be copied or archived.
pattern A pattern matching one or more pathnames of archive
members. A pattern must be given in the name-generating
notation of the pattern matching notation in Section
2.13, Pattern Matching Notation, including the filename
expansion rules in Section 2.13.3, Patterns Used for
Filename Expansion. The default, if no pattern is
specified, is to select all members in the archive.
In write mode, the standard input shall be used only if no file
operands are specified. It shall be a file containing a list of
pathnames, each terminated by a <newline> character.
In list and read modes, if -f is not specified, the standard
input shall be an archive file.
Otherwise, the standard input shall not be used.
The input file named by the archive option-argument, or standard
input when the archive is read from there, shall be a file
formatted according to one of the specifications in the EXTENDED
DESCRIPTION section or some other implementation-defined format.
The file /dev/tty shall be used to write prompts and read
responses.
The following environment variables shall affect the execution of
pax:
LANG Provide a default value for the internationalization
variables that are unset or null. (See the Base
Definitions volume of POSIX.1‐2017, Section 8.2,
Internationalization Variables the precedence of
internationalization variables used to determine the
values of locale categories.)
LC_ALL If set to a non-empty string value, override the values
of all the other internationalization variables.
LC_COLLATE
Determine the locale for the behavior of ranges,
equivalence classes, and multi-character collating
elements used in the pattern matching expressions for
the pattern operand, the basic regular expression for
the -s option, and the extended regular expression
defined for the yesexpr locale keyword in the
LC_MESSAGES category.
LC_CTYPE Determine the locale for the interpretation of
sequences of bytes of text data as characters (for
example, single-byte as opposed to multi-byte
characters in arguments and input files), the behavior
of character classes used in the extended regular
expression defined for the yesexpr locale keyword in
the LC_MESSAGES category, and pattern matching.
LC_MESSAGES
Determine the locale used to process affirmative
responses, and the locale used to affect the format and
contents of diagnostic messages and prompts written to
standard error.
LC_TIME Determine the format and contents of date and time
strings when the -v option is specified.
NLSPATH Determine the location of message catalogs for the
processing of LC_MESSAGES.
TMPDIR Determine the pathname that provides part of the
default global extended header record file, as
described for the -o globexthdr= keyword in the OPTIONS
section.
TZ Determine the timezone used to calculate date and time
strings when the -v option is specified. If TZ is unset
or null, an unspecified default timezone shall be used.
Default.
In write mode, if -f is not specified, the standard output shall
be the archive formatted according to one of the specifications
in the EXTENDED DESCRIPTION section, or some other
implementation-defined format (see -x format).
In list mode, when the -olistopt=format has been specified, the
selected archive members shall be written to standard output
using the format described under List Mode Format Specifications.
In list mode without the -olistopt=format option, the table of
contents of the selected archive members shall be written to
standard output using the following format:
"%s\n", <pathname>
If the -v option is specified in list mode, the table of contents
of the selected archive members shall be written to standard
output using the following formats.
For pathnames representing hard links to previous members of the
archive:
"%s == %s\n", <ls -l listing>, <linkname>
For all other pathnames:
"%s\n", <ls -l listing>
where <ls -l listing> shall be the format specified by the ls
utility with the -l option. When writing pathnames in this
format, it is unspecified what is written for fields for which
the underlying archive format does not have the correct
information, although the correct number of <blank>-separated
fields shall be written.
In list mode, standard output shall not be buffered more than a
pathname (plus any associated information and a <newline>
terminator) at a time.
If -v is specified in read, write, or copy modes, pax shall write
the pathnames it processes to the standard error output using the
following format:
"%s\n", <pathname>
These pathnames shall be written as soon as processing is begun
on the file or archive member, and shall be flushed to standard
error. The trailing <newline>, which shall not be buffered, is
written when the file has been read or written.
If the -s option is specified, and the replacement string has a
trailing 'p', substitutions shall be written to standard error in
the following format:
"%s >> %s\n", <original pathname>, <new pathname>
In all operating modes of pax, optional messages of unspecified
format concerning the input archive format and volume number, the
number of files, blocks, volumes, and media parts as well as
other diagnostic messages may be written to standard error.
In all formats, for both standard output and standard error, it
is unspecified how non-printable characters in pathnames or link
names are written.
When using the -xpax archive format, if a filename, link name,
group name, owner name, or any other field in an extended header
record cannot be translated between the codeset in use for that
extended header record and the character set of the current
locale, pax shall write a diagnostic message to standard error,
shall process the file as described for the -o invalid= option,
and then shall continue processing with the next file.
In read mode, the extracted output files shall be of the archived
file type. In copy mode, the copied output files shall be the
type of the file being copied. In either mode, existing files in
the destination hierarchy shall be overwritten only when all
permission (-p), modification time (-u), and invalid-value
(-oinvalid=) tests allow it.
In write mode, the output file named by the -f option-argument
shall be a file formatted according to one of the specifications
in the EXTENDED DESCRIPTION section, or some other
implementation-defined format.
pax Interchange Format
A pax archive tape or file produced in the -xpax format shall
contain a series of blocks. The physical layout of the archive
shall be identical to the ustar format described in ustar
Interchange Format. Each file archived shall be represented by
the following sequence:
* An optional header block with extended header records. This
header block is of the form described in pax Header Block,
with a typeflag value of x or g. The extended header
records, described in pax Extended Header, shall be included
as the data for this header block.
* A header block that describes the file. Any fields in the
preceding optional extended header shall override the
associated fields in this header block for this file.
* Zero or more blocks that contain the contents of the file.
At the end of the archive file there shall be two 512-byte blocks
filled with binary zeros, interpreted as an end-of-archive
indicator.
A schematic of an example archive with global extended header
records and two actual files is shown in Figure 4-1, pax Format
Archive Example. In the example, the second file in the archive
has no extended header preceding it, presumably because it has no
need for extended attributes.
Figure 4-1: pax Format Archive Example
pax Header Block
The pax header block shall be identical to the ustar header block
described in ustar Interchange Format, except that two additional
typeflag values are defined:
x Represents extended header records for the following file
in the archive (which shall have its own ustar header
block). The format of these extended header records shall
be as described in pax Extended Header.
g Represents global extended header records for the following
files in the archive. The format of these extended header
records shall be as described in pax Extended Header. Each
value shall affect all subsequent files that do not
override that value in their own extended header record and
until another global extended header record is reached that
provides another value for the same field. The typeflag g
global headers should not be used with interchange media
that could suffer partial data loss in transporting the
archive.
For both of these types, the size field shall be the size of the
extended header records in octets. The other fields in the header
block are not meaningful to this version of the pax utility.
However, if this archive is read by a pax utility conforming to
the ISO POSIX‐2:1993 standard, the header block fields are used
to create a regular file that contains the extended header
records as data. Therefore, header block field values should be
selected to provide reasonable file access to this regular file.
A further difference from the ustar header block is that data
blocks for files of typeflag 1 (the digit one) (hard link) may be
included, which means that the size field may be greater than
zero. Archives created by pax -o linkdata shall include these
data blocks with the hard links.
pax Extended Header
A pax extended header contains values that are inappropriate for
the ustar header block because of limitations in that format:
fields requiring a character encoding other than that described
in the ISO/IEC 646:1991 standard, fields representing file
attributes not described in the ustar header, and fields whose
format or length do not fit the requirements of the ustar header.
The values in an extended header add attributes to the following
file (or files; see the description of the typeflag g header
block) or override values in the following header block(s), as
indicated in the following list of keywords.
An extended header shall consist of one or more records, each
constructed as follows:
"%d %s=%s\n", <length>, <keyword>, <value>
The extended header records shall be encoded according to the
ISO/IEC 10646‐1:2000 standard UTF‐8 encoding. The <length> field,
<blank>, <equals-sign>, and <newline> shown shall be limited to
the portable character set, as encoded in UTF‐8. The <keyword>
fields can be any UTF‐8 characters. The <length> field shall be
the decimal length of the extended header record in octets,
including the trailing <newline>. If there is a hdrcharset
extended header in effect for a file, the value field for any
gname, linkpath, path, and uname extended header records shall be
encoded using the character set specified by the hdrcharset
extended header record; otherwise, the value field shall be
encoded using UTF‐8. The value field for all other keywords
specified by POSIX.1‐2008 shall be encoded using UTF‐8.
The <keyword> field shall be one of the entries from the
following list or a keyword provided as an implementation
extension. Keywords consisting entirely of lowercase letters,
digits, and periods are reserved for future standardization. A
keyword shall not include an <equals-sign>. (In the following
list, the notations ``file(s)'' or ``block(s)'' is used to
acknowledge that a keyword affects the following single file
after a typeflag x extended header, but possibly multiple files
after typeflag g. Any requirements in the list for pax to
include a record when in write or copy mode shall apply only when
such a record has not already been provided through the use of
the -o option. When used in copy mode, pax shall behave as if an
archive had been created with applicable extended header records
and then extracted.)
atime The file access time for the following file(s),
equivalent to the value of the st_atime member of the
stat structure for a file, as described by the stat()
function. The access time shall be restored if the
process has appropriate privileges required to do so.
The format of the <value> shall be as described in pax
Extended Header File Times.
charset The name of the character set used to encode the data
in the following file(s). The entries in the following
table are defined to refer to known standards;
additional names may be agreed on between the
originator and recipient.
┌─────────────────────────┬───────────────────────────────┐
│ <value> │ Formal Standard │
├─────────────────────────┼───────────────────────────────┤
│ ISO-IR 646 1990 │ ISO/IEC 646:1990 │
│ ISO-IR 8859 1 1998 │ ISO/IEC 8859‐1:1998 │
│ ISO-IR 8859 2 1999 │ ISO/IEC 8859‐2:1999 │
│ ISO-IR 8859 3 1999 │ ISO/IEC 8859‐3:1999 │
│ ISO-IR 8859 4 1998 │ ISO/IEC 8859‐4:1998 │
│ ISO-IR 8859 5 1999 │ ISO/IEC 8859‐5:1999 │
│ ISO-IR 8859 6 1999 │ ISO/IEC 8859‐6:1999 │
│ ISO-IR 8859 7 1987 │ ISO/IEC 8859‐7:1987 │
│ ISO-IR 8859 8 1999 │ ISO/IEC 8859‐8:1999 │
│ ISO-IR 8859 9 1999 │ ISO/IEC 8859‐9:1999 │
│ ISO-IR 8859 10 1998 │ ISO/IEC 8859‐10:1998 │
│ ISO-IR 8859 13 1998 │ ISO/IEC 8859‐13:1998 │
│ ISO-IR 8859 14 1998 │ ISO/IEC 8859‐14:1998 │
│ ISO-IR 8859 15 1999 │ ISO/IEC 8859‐15:1999 │
│ ISO-IR 10646 2000 │ ISO/IEC 10646:2000 │
│ ISO-IR 10646 2000 UTF-8 │ ISO/IEC 10646, UTF-8 encoding │
│ BINARY │ None. │
└─────────────────────────┴───────────────────────────────┘
The encoding is included in an extended header for
information only; when pax is used as described in
POSIX.1‐2008, it shall not translate the file data into
any other encoding. The BINARY entry indicates
unencoded binary data.
When used in write or copy mode, it is implementation-
defined whether pax includes a charset extended header
record for a file.
comment A series of characters used as a comment. All
characters in the <value> field shall be ignored by
pax.
gid The group ID of the group that owns the file, expressed
as a decimal number using digits from the
ISO/IEC 646:1991 standard. This record shall override
the gid field in the following header block(s). When
used in write or copy mode, pax shall include a gid
extended header record for each file whose group ID is
greater than 2097151 (octal 7777777).
gname The group of the file(s), formatted as a group name in
the group database. This record shall override the gid
and gname fields in the following header block(s), and
any gid extended header record. When used in read,
copy, or list mode, pax shall translate the name from
the encoding in the header record to the character set
appropriate for the group database on the receiving
system. If any of the characters cannot be translated,
and if neither the -oinvalid=UTF‐8 option nor the
-oinvalid=binary option is specified, the results are
implementation-defined. When used in write or copy
mode, pax shall include a gname extended header record
for each file whose group name cannot be represented
entirely with the letters and digits of the portable
character set.
hdrcharset
The name of the character set used to encode the value
field of the gname, linkpath, path, and uname pax
extended header records. The entries in the following
table are defined to refer to known standards;
additional names may be agreed between the originator
and the recipient.
┌─────────────────────────┬───────────────────────────────┐
│ <value> │ Formal Standard │
├─────────────────────────┼───────────────────────────────┤
│ ISO-IR 10646 2000 UTF-8 │ ISO/IEC 10646, UTF-8 encoding │
│ BINARY │ None. │
└─────────────────────────┴───────────────────────────────┘
If no hdrcharset extended header record is specified,
the default character set used to encode all values in
extended header records shall be the
ISO/IEC 10646‐1:2000 standard UTF‐8 encoding.
The BINARY entry indicates that all values recorded in
extended headers for affected files are unencoded
binary data from the underlying system.
linkpath The pathname of a link being created to another file,
of any type, previously archived. This record shall
override the linkname field in the following ustar
header block(s). The following ustar header block shall
determine the type of link created. If typeflag of the
following header block is 1, it shall be a hard link.
If typeflag is 2, it shall be a symbolic link and the
linkpath value shall be the contents of the symbolic
link. The pax utility shall translate the name of the
link (contents of the symbolic link) from the encoding
in the header to the character set appropriate for the
local file system. When used in write or copy mode, pax
shall include a linkpath extended header record for
each link whose pathname cannot be represented entirely
with the members of the portable character set other
than NUL.
mtime The file modification time of the following file(s),
equivalent to the value of the st_mtime member of the
stat structure for a file, as described in the stat()
function. This record shall override the mtime field in
the following header block(s). The modification time
shall be restored if the process has appropriate
privileges required to do so. The format of the <value>
shall be as described in pax Extended Header File
Times.
path The pathname of the following file(s). This record
shall override the name and prefix fields in the
following header block(s). The pax utility shall
translate the pathname of the file from the encoding in
the header to the character set appropriate for the
local file system.
When used in write or copy mode, pax shall include a
path extended header record for each file whose
pathname cannot be represented entirely with the
members of the portable character set other than NUL.
realtime.any
The keywords prefixed by ``realtime.'' are reserved for
future standardization.
security.any
The keywords prefixed by ``security.'' are reserved for
future standardization.
size The size of the file in octets, expressed as a decimal
number using digits from the ISO/IEC 646:1991 standard.
This record shall override the size field in the
following header block(s). When used in write or copy
mode, pax shall include a size extended header record
for each file with a size value greater than 8589934591
(octal 77777777777).
uid The user ID of the file owner, expressed as a decimal
number using digits from the ISO/IEC 646:1991 standard.
This record shall override the uid field in the
following header block(s). When used in write or copy
mode, pax shall include a uid extended header record
for each file whose owner ID is greater than 2097151
(octal 7777777).
uname The owner of the following file(s), formatted as a user
name in the user database. This record shall override
the uid and uname fields in the following header
block(s), and any uid extended header record. When used
in read, copy, or list mode, pax shall translate the
name from the encoding in the header record to the
character set appropriate for the user database on the
receiving system. If any of the characters cannot be
translated, and if neither the -oinvalid=UTF‐8 option
nor the -oinvalid=binary option is specified, the
results are implementation-defined. When used in write
or copy mode, pax shall include a uname extended header
record for each file whose user name cannot be
represented entirely with the letters and digits of the
portable character set.
If the <value> field is zero length, it shall delete any header
block field, previously entered extended header value, or global
extended header value of the same name.
If a keyword in an extended header record (or in a -o option-
argument) overrides or deletes a corresponding field in the ustar
header block, pax shall ignore the contents of that header block
field.
Unlike the ustar header block fields, NULs shall not delimit
<value>s; all characters within the <value> field shall be
considered data for the field. None of the length limitations of
the ustar header block fields in Table 4-14, ustar Header Block
shall apply to the extended header records.
pax Extended Header Keyword Precedence
This section describes the precedence in which the various header
records and fields and command line options are selected to apply
to a file in the archive. When pax is used in read or list modes,
it shall determine a file attribute in the following sequence:
1. If -odelete=keyword-prefix is used, the affected attributes
shall be determined from step 7., if applicable, or ignored
otherwise.
2. If -okeyword:= is used, the affected attributes shall be
ignored.
3. If -okeyword:=value is used, the affected attribute shall be
assigned the value.
4. If there is a typeflag x extended header record, the affected
attribute shall be assigned the <value>. When extended header
records conflict, the last one given in the header shall take
precedence.
5. If -okeyword=value is used, the affected attribute shall be
assigned the value.
6. If there is a typeflag g global extended header record, the
affected attribute shall be assigned the <value>. When global
extended header records conflict, the last one given in the
global header shall take precedence.
7. Otherwise, the attribute shall be determined from the ustar
header block.
pax Extended Header File Times
The pax utility shall write an mtime record for each file in
write or copy modes if the file's modification time cannot be
represented exactly in the ustar header logical record described
in ustar Interchange Format. This can occur if the time is out
of ustar range, or if the file system of the underlying
implementation supports non-integer time granularities and the
time is not an integer. All of these time records shall be
formatted as a decimal representation of the time in seconds
since the Epoch. If a <period> ('.') decimal point character is
present, the digits to the right of the point shall represent the
units of a subsecond timing granularity, where the first digit is
tenths of a second and each subsequent digit is a tenth of the
previous digit. In read or copy mode, the pax utility shall
truncate the time of a file to the greatest value that is not
greater than the input header file time. In write or copy mode,
the pax utility shall output a time exactly if it can be
represented exactly as a decimal number, and otherwise shall
generate only enough digits so that the same time shall be
recovered if the file is extracted on a system whose underlying
implementation supports the same time granularity.
ustar Interchange Format
A ustar archive tape or file shall contain a series of logical
records. Each logical record shall be a fixed-size logical record
of 512 octets (see below). Although this format may be thought of
as being stored on 9-track industry-standard 12.7 mm (0.5 in)
magnetic tape, other types of transportable media are not
excluded. Each file archived shall be represented by a header
logical record that describes the file, followed by zero or more
logical records that give the contents of the file. At the end of
the archive file there shall be two 512-octet logical records
filled with binary zeros, interpreted as an end-of-archive
indicator.
The logical records may be grouped for physical I/O operations,
as described under the -bblocksize and -x ustar options. Each
group of logical records may be written with a single operation
equivalent to the write() function. On magnetic tape, the result
of this write shall be a single tape physical block. The last
physical block shall always be the full size, so logical records
after the two zero logical records may contain undefined data.
The header logical record shall be structured as shown in the
following table. All lengths and offsets are in decimal.
Table 4-14: ustar Header Block
┌────────────┬──────────────┬────────────────────┐
│ Field Name │ Octet Offset │ Length (in Octets) │
├────────────┼──────────────┼────────────────────┤
│ name │ 0 │ 100 │
│ mode │ 100 │ 8 │
│ uid │ 108 │ 8 │
│ gid │ 116 │ 8 │
│ size │ 124 │ 12 │
│ mtime │ 136 │ 12 │
│ chksum │ 148 │ 8 │
│ typeflag │ 156 │ 1 │
│ linkname │ 157 │ 100 │
│ magic │ 257 │ 6 │
│ version │ 263 │ 2 │
│ uname │ 265 │ 32 │
│ gname │ 297 │ 32 │
│ devmajor │ 329 │ 8 │
│ devminor │ 337 │ 8 │
│ prefix │ 345 │ 155 │
└────────────┴──────────────┴────────────────────┘
All characters in the header logical record shall be represented
in the coded character set of the ISO/IEC 646:1991 standard. For
maximum portability between implementations, names should be
selected from characters represented by the portable filename
character set as octets with the most significant bit zero. If an
implementation supports the use of characters outside of <slash>
and the portable filename character set in names for files,
users, and groups, one or more implementation-defined encodings
of these characters shall be provided for interchange purposes.
However, the pax utility shall never create filenames on the
local system that cannot be accessed via the procedures described
in POSIX.1‐2008. If a filename is found on the medium that would
create an invalid filename, it is implementation-defined whether
the data from the file is stored on the file hierarchy and under
what name it is stored. The pax utility may choose to ignore
these files as long as it produces an error indicating that the
file is being ignored.
Each field within the header logical record is contiguous; that
is, there is no padding used. Each character on the archive
medium shall be stored contiguously.
The fields magic, uname, and gname are character strings each
terminated by a NUL character. The fields name, linkname, and
prefix are NUL-terminated character strings except when all
characters in the array contain non-NUL characters including the
last character. The version field is two octets containing the
characters "00" (zero-zero). The typeflag contains a single
character. All other fields are leading zero-filled octal numbers
using digits from the ISO/IEC 646:1991 standard IRV. Each numeric
field is terminated by one or more <space> or NUL characters.
The name and the prefix fields shall produce the pathname of the
file. A new pathname shall be formed, if prefix is not an empty
string (its first character is not NUL), by concatenating prefix
(up to the first NUL character), a <slash> character, and name;
otherwise, name is used alone. In either case, name is terminated
at the first NUL character. If prefix begins with a NUL
character, it shall be ignored. In this manner, pathnames of at
most 256 characters can be supported. If a pathname does not fit
in the space provided, pax shall notify the user of the error,
and shall not store any part of the file—header or data—on the
medium.
The linkname field, described below, shall not use the prefix to
produce a pathname. As such, a linkname is limited to 100
characters. If the name does not fit in the space provided, pax
shall notify the user of the error, and shall not attempt to
store the link on the medium.
The mode field provides 12 bits encoded in the ISO/IEC 646:1991
standard octal digit representation. The encoded bits shall
represent the following values:
Table: ustar mode Field
┌───────────┬──────────────────┬─────────────────────────────────────────────────┐
│ Bit Value │ POSIX.1‐2008 Bit │ Description │
├───────────┼──────────────────┼─────────────────────────────────────────────────┤
│ 04000 │ S_ISUID │ Set UID on execution. │
│ 02000 │ S_ISGID │ Set GID on execution. │
│ 01000 │ <reserved> │ Reserved for future standardization. │
│ 00400 │ S_IRUSR │ Read permission for file owner class. │
│ 00200 │ S_IWUSR │ Write permission for file owner class. │
│ 00100 │ S_IXUSR │ Execute/search permission for file owner class. │
│ 00040 │ S_IRGRP │ Read permission for file group class. │
│ 00020 │ S_IWGRP │ Write permission for file group class. │
│ 00010 │ S_IXGRP │ Execute/search permission for file group class. │
│ 00004 │ S_IROTH │ Read permission for file other class. │
│ 00002 │ S_IWOTH │ Write permission for file other class. │
│ 00001 │ S_IXOTH │ Execute/search permission for file other class. │
└───────────┴──────────────────┴─────────────────────────────────────────────────┘
When appropriate privileges are required to set one of these mode
bits, and the user restoring the files from the archive does not
have appropriate privileges, the mode bits for which the user
does not have appropriate privileges shall be ignored. Some of
the mode bits in the archive format are not mentioned elsewhere
in this volume of POSIX.1‐2017. If the implementation does not
support those bits, they may be ignored.
The uid and gid fields are the user and group ID of the owner and
group of the file, respectively.
The size field is the size of the file in octets. If the typeflag
field is set to specify a file to be of type 1 (a link) or 2 (a
symbolic link), the size field shall be specified as zero. If the
typeflag field is set to specify a file of type 5 (directory),
the size field shall be interpreted as described under the
definition of that record type. No data logical records are
stored for types 1, 2, or 5. If the typeflag field is set to 3
(character special file), 4 (block special file), or 6 (FIFO),
the meaning of the size field is unspecified by this volume of
POSIX.1‐2017, and no data logical records shall be stored on the
medium. Additionally, for type 6, the size field shall be ignored
when reading. If the typeflag field is set to any other value,
the number of logical records written following the header shall
be (size+511)/512, ignoring any fraction in the result of the
division.
The mtime field shall be the modification time of the file at the
time it was archived. It is the ISO/IEC 646:1991 standard
representation of the octal value of the modification time
obtained from the stat() function.
The chksum field shall be the ISO/IEC 646:1991 standard IRV
representation of the octal value of the simple sum of all octets
in the header logical record. Each octet in the header shall be
treated as an unsigned value. These values shall be added to an
unsigned integer, initialized to zero, the precision of which is
not less than 17 bits. When calculating the checksum, the chksum
field is treated as if it were all <space> characters.
The typeflag field specifies the type of file archived. If a
particular implementation does not recognize the type, or the
user does not have appropriate privileges to create that type,
the file shall be extracted as if it were a regular file if the
file type is defined to have a meaning for the size field that
could cause data logical records to be written on the medium (see
the previous description for size). If conversion to a regular
file occurs, the pax utility shall produce an error indicating
that the conversion took place. All of the typeflag fields shall
be coded in the ISO/IEC 646:1991 standard IRV:
0 Represents a regular file. For backwards-compatibility, a
typeflag value of binary zero ('\0') should be recognized
as meaning a regular file when extracting files from the
archive. Archives written with this version of the
archive file format create regular files with a typeflag
value of the ISO/IEC 646:1991 standard IRV '0'.
1 Represents a file linked to another file, of any type,
previously archived. Such files are identified by having
the same device and file serial numbers, and pathnames
that refer to different directory entries. All such files
shall be archived as linked files. The linked-to name is
specified in the linkname field with a NUL-character
terminator if it is less than 100 octets in length.
2 Represents a symbolic link. The contents of the symbolic
link shall be stored in the linkname field.
3,4 Represent character special files and block special files
respectively. In this case the devmajor and devminor
fields shall contain information defining the device, the
format of which is unspecified by this volume of
POSIX.1‐2017. Implementations may map the device
specifications to their own local specification or may
ignore the entry.
5 Specifies a directory or subdirectory. On systems where
disk allocation is performed on a directory basis, the
size field shall contain the maximum number of octets
(which may be rounded to the nearest disk block
allocation unit) that the directory may hold. A size
field of zero indicates no such limiting. Systems that do
not support limiting in this manner should ignore the
size field.
6 Specifies a FIFO special file. Note that the archiving of
a FIFO file archives the existence of this file and not
its contents.
7 Reserved to represent a file to which an implementation
has associated some high-performance attribute.
Implementations without such extensions should treat this
file as a regular file (type 0).
A‐Z The letters 'A' to 'Z', inclusive, are reserved for
custom implementations. All other values are reserved for
future versions of this standard.
It is unspecified whether files with pathnames that refer to the
same directory entry are archived as linked files or as separate
files. If they are archived as linked files, this means that
attempting to extract both pathnames from the resulting archive
will always cause an error (unless the -u option is used) because
the link cannot be created.
It is unspecified whether files with the same device and file
serial numbers being appended to an archive are treated as linked
files to members that were in the archive before the append.
Attempts to archive a socket shall produce a diagnostic message
when ustar interchange format is used, but may be allowed when
pax interchange format is used. Handling of other file types is
implementation-defined.
The magic field is the specification that this archive was output
in this archive format. If this field contains ustar (the five
characters from the ISO/IEC 646:1991 standard IRV shown followed
by NUL), the uname and gname fields shall contain the
ISO/IEC 646:1991 standard IRV representation of the owner and
group of the file, respectively (truncated to fit, if necessary).
When the file is restored by a privileged, protection-preserving
version of the utility, the user and group databases shall be
scanned for these names. If found, the user and group IDs
contained within these files shall be used rather than the values
contained within the uid and gid fields.
cpio Interchange Format
The octet-oriented cpio archive format shall be a series of
entries, each comprising a header that describes the file, the
name of the file, and then the contents of the file.
An archive may be recorded as a series of fixed-size blocks of
octets. This blocking shall be used only to make physical I/O
more efficient. The last group of blocks shall always be at the
full size.
For the octet-oriented cpio archive format, the individual entry
information shall be in the order indicated and described by the
following table; see also the <cpio.h> header.
Table 4-16: Octet-Oriented cpio Archive Entry
┌──────────────────────┬────────────────────┬─────────────────┐
│ Header Field Name │ Length (in Octets) │ Interpreted as │
├──────────────────────┼────────────────────┼─────────────────┤
│ c_magic │ 6 │ Octal number │
│ c_dev │ 6 │ Octal number │
│ c_ino │ 6 │ Octal number │
│ c_mode │ 6 │ Octal number │
│ c_uid │ 6 │ Octal number │
│ c_gid │ 6 │ Octal number │
│ c_nlink │ 6 │ Octal number │
│ c_rdev │ 6 │ Octal number │
│ c_mtime │ 11 │ Octal number │
│ c_namesize │ 6 │ Octal number │
│ c_filesize │ 11 │ Octal number │
├──────────────────────┼────────────────────┼─────────────────┤
│ Filename Field Name │ Length │ Interpreted as │
├──────────────────────┴────────────────────┴─────────────────┤
│ c_name c_namesize Pathname string │
├──────────────────────┬────────────────────┬─────────────────┤
│ File Data Field Name │ Length │ Interpreted as │
├──────────────────────┴────────────────────┴─────────────────┤
│ c_filedata c_filesize Data │
└─────────────────────────────────────────────────────────────┘
cpio Header
For each file in the archive, a header as defined previously
shall be written. The information in the header fields is written
as streams of the ISO/IEC 646:1991 standard characters
interpreted as octal numbers. The octal numbers shall be extended
to the necessary length by appending the ISO/IEC 646:1991
standard IRV zeros at the most-significant-digit end of the
number; the result is written to the most-significant digit of
the stream of octets first. The fields shall be interpreted as
follows:
c_magic Identify the archive as being a transportable archive
by containing the identifying value "070707".
c_dev, c_ino
Contains values that uniquely identify the file within
the archive (that is, no files contain the same pair of
c_dev and c_ino values unless they are links to the
same file). The values shall be determined in an
unspecified manner.
c_mode Contains the file type and access permissions as
defined in the following table.
Table 4-17: Values for cpio c_mode Field
───┬───────────────────────┬─────────┬────────────────────── │
F│ile Permissions Name │Value │ Indicates │
───┼───────────────────────┼─────────┼────────────────────── │
C│_IRUSR │000400 R│ead by owner │
C│_IWUSR │000200 W│rite by owner │
C│_IXUSR │000100 E│xecute by owner │
C│_IRGRP │000040 R│ead by group │
C│_IWGRP │000020 W│rite by group │
C│_IXGRP │000010 E│xecute by group │
C│_IROTH │000004 R│ead by others │
C│_IWOTH │000002 W│rite by others │
C│_IXOTH │000001 E│xecute by others │
C│_ISUID │004000 S│et uid │
C│_ISGID │002000 S│et gid │
C│_ISVTX │001000 R│eserved │
───┼───────────────────────┼─────────┼────────────────────── │
│ File Type Name │Value │ Indicates │
───┼───────────────────────┼─────────┼────────────────────── │
C│_ISDIR │040000 D│irectory │
C│_ISFIFO │010000 F│IFO │
C│_ISREG 0│100000 R│egular file │
C│_ISLNK 0│120000 S│ymbolic link │
│ │ │ │
│ C_ISBLK │ 060000 │ Block special file │
│ C_ISCHR │ 020000 │ Character special file │
│ C_ISSOCK │ 0140000 │ Socket │
│ │ │ │
│ C_ISCTG │ 0110000 │ Reserved │
└───────────────────────┴─────────┴────────────────────────┘
Directories, FIFOs, symbolic links, and regular files
shall be supported on a system conforming to this
volume of POSIX.1‐2017; additional values defined
previously are reserved for compatibility with existing
systems. Additional file types may be supported;
however, such files should not be written to archives
intended to be transported to other systems.
c_uid Contains the user ID of the owner.
c_gid Contains the group ID of the group.
c_nlink Contains a number greater than or equal to the number
of links in the archive referencing the file. If the -a
option is used to append to a cpio archive, then the
pax utility need not account for the files in the
existing part of the archive when calculating the
c_nlink values for the appended part of the archive,
and need not alter the c_nlink values in the existing
part of the archive if additional files with the same
c_dev and c_ino values are appended to the archive.
c_rdev Contains implementation-defined information for
character or block special files.
c_mtime Contains the latest time of modification of the file at
the time the archive was created.
c_namesize
Contains the length of the pathname, including the
terminating NUL character.
c_filesize
Contains the length in octets of the data section
following the header structure.
cpio Filename
The c_name field shall contain the pathname of the file. The
length of this field in octets is the value of c_namesize.
If a filename is found on the medium that would create an invalid
pathname, it is implementation-defined whether the data from the
file is stored on the file hierarchy and under what name it is
stored.
All characters shall be represented in the ISO/IEC 646:1991
standard IRV. For maximum portability between implementations,
names should be selected from characters represented by the
portable filename character set as octets with the most
significant bit zero. If an implementation supports the use of
characters outside the portable filename character set in names
for files, users, and groups, one or more implementation-defined
encodings of these characters shall be provided for interchange
purposes. However, the pax utility shall never create filenames
on the local system that cannot be accessed via the procedures
described previously in this volume of POSIX.1‐2017. If a
filename is found on the medium that would create an invalid
filename, it is implementation-defined whether the data from the
file is stored on the local file system and under what name it is
stored. The pax utility may choose to ignore these files as long
as it produces an error indicating that the file is being
ignored.
cpio File Data
Following c_name, there shall be c_filesize octets of data.
Interpretation of such data occurs in a manner dependent on the
file. For regular files, the data shall consist of the contents
of the file. For symbolic links, the data shall consist of the
contents of the symbolic link. If c_filesize is zero, no data
shall be contained in c_filedata.
When restoring from an archive:
* If the user does not have appropriate privileges to create a
file of the specified type, pax shall ignore the entry and
write an error message to standard error.
* Only regular files and symbolic links have data to be
restored. Presuming a regular file meets any selection
criteria that might be imposed on the format-reading utility
by the user, such data shall be restored.
* If a user does not have appropriate privileges to set a
particular mode flag, the flag shall be ignored. Some of the
mode flags in the archive format are not mentioned elsewhere
in this volume of POSIX.1‐2017. If the implementation does
not support those flags, they may be ignored.
cpio Special Entries
FIFO special files, directories, and the trailer shall be
recorded with c_filesize equal to zero. Symbolic links shall be
recorded with c_filesize equal to the length of the contents of
the symbolic link. For other special files, c_filesize is
unspecified by this volume of POSIX.1‐2017. The header for the
next file entry in the archive shall be written directly after
the last octet of the file entry preceding it. A header denoting
the filename TRAILER!!! shall indicate the end of the archive;
the contents of octets in the last block of the archive following
such a header are undefined.
The following exit values shall be returned:
0 All files were processed successfully.
>0 An error occurred.
If pax cannot create a file or a link when reading an archive or
cannot find a file when writing an archive, or cannot preserve
the user ID, group ID, or file mode when the -p option is
specified, a diagnostic message shall be written to standard
error and a non-zero exit status shall be returned, but
processing shall continue. In the case where pax cannot create a
link to a file, pax shall not, by default, create a second copy
of the file.
If the extraction of a file from an archive is prematurely
terminated by a signal or error, pax may have only partially
extracted the file or (if the -n option was not specified) may
have extracted a file of the same name as that specified by the
user, but which is not the file the user wanted. Additionally,
the file modes of extracted directories may have additional bits
from the S_IRWXU mask set as well as incorrect modification and
access times.
The following sections are informative.
Caution is advised when using the -a option to append to a cpio
format archive. If any of the files being appended happen to be
given the same c_dev and c_ino values as a file in the existing
part of the archive, then they may be treated as links to that
file on extraction. Thus, it is risky to use -a with cpio format
except when it is done on the same system that the original
archive was created on, and with the same pax utility, and in the
knowledge that there has been little or no file system activity
since the original archive was created that could lead to any of
the files appended being given the same c_dev and c_ino values as
an unrelated file in the existing part of the archive. Also, when
(intentionally) appending additional links to a file in the
existing part of the archive, the c_nlink values in the modified
archive can be smaller than the number of links to the file in
the archive, which may mean that the links are not preserved on
extraction.
The -p (privileges) option was invented to reconcile differences
between historical tar and cpio implementations. In particular,
the two utilities use -m in diametrically opposed ways. The -p
option also provides a consistent means of extending the ways in
which future file attributes can be addressed, such as for
enhanced security systems or high-performance files. Although it
may seem complex, there are really two modes that are most
commonly used:
-p e ``Preserve everything''. This would be used by the
historical superuser, someone with all appropriate
privileges, to preserve all aspects of the files as they
are recorded in the archive. The e flag is the sum of o
and p, and other implementation-defined attributes.
-p p ``Preserve'' the file mode bits. This would be used by
the user with regular privileges who wished to preserve
aspects of the file other than the ownership. The file
times are preserved by default, but two other flags are
offered to disable these and use the time of extraction.
The one pathname per line format of standard input precludes
pathnames containing <newline> characters. Although such
pathnames violate the portable filename guidelines, they may
exist and their presence may inhibit usage of pax within shell
scripts. This problem is inherited from historical archive
programs. The problem can be avoided by listing filename
arguments on the command line instead of on standard input.
It is almost certain that appropriate privileges are required for
pax to accomplish parts of this volume of POSIX.1‐2017.
Specifically, creating files of type block special or character
special, restoring file access times unless the files are owned
by the user (the -t option), or preserving file owner, group, and
mode (the -p option) all probably require appropriate privileges.
In read mode, implementations are permitted to overwrite files
when the archive has multiple members with the same name. This
may fail if permissions on the first version of the file do not
permit it to be overwritten.
The cpio and ustar formats can only support files up to
8589934592 bytes (8 ∗ 2^30) in size.
When archives containing binary header information are listed ,
the filenames printed may cause strange behavior on some
terminals.
When all of the following are true:
1. A file of type directory is being placed into an archive.
2. The ustar archive format is being used.
3. The pathname of the directory is less than or equal to 155
bytes long (it will fit in the prefix field in the ustar
header block).
4. The last component of the pathname of the directory is longer
than 100 bytes long (it will not fit in the name field in the
ustar header block).
some implementations of the pax utility will place the entire
directory pathname in the prefix field, set the name field to an
empty string, and place the directory in the archive. Other
implementations of the pax utility will give an error under these
conditions because the name field is not large enough to hold the
last component of the directory name. This standard allows
either behavior. However, when extracting a directory from a
ustar format archive, this standard requires that all
implementations be able to extract a directory even if the name
field contains an empty string as long as the prefix field does
not also contain an empty string.
The following command:
pax -w -f /dev/rmt/1m .
copies the contents of the current directory to tape drive 1,
medium density (assuming historical System V device naming
procedures—the historical BSD device name would be /dev/rmt9).
The following commands:
mkdir newdir
pax -rw olddir newdir
copy the olddir directory hierarchy to newdir.
pax -r -s ',^//*usr//*,,' -f a.pax
reads the archive a.pax, with all files rooted in /usr in the
archive extracted relative to the current directory.
Using the option:
-o listopt="%M %(atime)T %(size)D %(name)s"
overrides the default output description in Standard Output and
instead writes:
-rw-rw--- Jan 12 15:53 2003 1492 /usr/foo/bar
Using the options:
-o listopt='%L\t%(size)D\n%.7' \
-o listopt='(name)s\n%(atime)T\n%T'
overrides the default output description in Standard Output and
instead writes:
/usr/foo/bar -> /tmp 1492
/usr/fo
Jan 12 15:53 1991
Jan 31 15:53 2003
The pax utility was new for the ISO POSIX‐2:1993 standard. It
represents a peaceful compromise between advocates of the
historical tar and cpio utilities.
A fundamental difference between cpio and tar was in the way
directories were treated. The cpio utility did not treat
directories differently from other files, and to select a
directory and its contents required that each file in the
hierarchy be explicitly specified. For tar, a directory matched
every file in the file hierarchy it rooted.
The pax utility offers both interfaces; by default, directories
map into the file hierarchy they root. The -d option causes pax
to skip any file not explicitly referenced, as cpio historically
did. The tar -style behavior was chosen as the default because it
was believed that this was the more common usage and because tar
is the more commonly available interface, as it was historically
provided on both System V and BSD implementations.
The data interchange format specification in this volume of
POSIX.1‐2017 requires that processes with ``appropriate
privileges'' shall always restore the ownership and permissions
of extracted files exactly as archived. If viewed from the
historic equivalence between superuser and ``appropriate
privileges'', there are two problems with this requirement.
First, users running as superusers may unknowingly set dangerous
permissions on extracted files. Second, it is needlessly
limiting, in that superusers cannot extract files and own them as
superuser unless the archive was created by the superuser. (It
should be noted that restoration of ownerships and permissions
for the superuser, by default, is historical practice in cpio,
but not in tar.) In order to avoid these two problems, the pax
specification has an additional ``privilege'' mechanism, the -p
option. Only a pax invocation with the privileges needed, and
which has the -p option set using the e specification character,
has appropriate privileges to restore full ownership and
permission information.
Note also that this volume of POSIX.1‐2017 requires that the file
ownership and access permissions shall be set, on extraction, in
the same fashion as the creat() function when provided with the
mode stored in the archive. This means that the file creation
mask of the user is applied to the file permissions.
Users should note that directories may be created by pax while
extracting files with permissions that are different from those
that existed at the time the archive was created. When extracting
sensitive information into a directory hierarchy that no longer
exists, users are encouraged to set their file creation mask
appropriately to protect these files during extraction.
The table of contents output is written to standard output to
facilitate pipeline processing.
An early proposal had hard links displaying for all pathnames.
This was removed because it complicates the output of the case
where -v is not specified and does not match historical cpio
usage. The hard-link information is available in the -v display.
The description of the -l option allows implementations to make
hard links to symbolic links. Earlier versions of this standard
did not specify any way to create a hard link to a symbolic link,
but many implementations provided this capability as an
extension. If there are hard links to symbolic links when an
archive is created, the implementation is required to archive the
hard link in the archive (unless -H or -L is specified). When in
read mode and in copy mode, implementations supporting hard links
to symbolic links should use them when appropriate.
The archive formats inherited from the POSIX.1‐1990 standard have
certain restrictions that have been brought along from historical
usage. For example, there are restrictions on the length of
pathnames stored in the archive. When pax is used in copy(-rw)
mode (copying directory hierarchies), the ability to use
extensions from the -xpax format overcomes these restrictions.
The default blocksize value of 5120 bytes for cpio was selected
because it is one of the standard block-size values for cpio, set
when the -B option is specified. (The other default block-size
value for cpio is 512 bytes, and this was considered to be too
small.) The default block value of 10240 bytes for tar was
selected because that is the standard block-size value for BSD
tar. The maximum block size of 32256 bytes (215-512 bytes) is
the largest multiple of 512 bytes that fits into a signed 16-bit
tape controller transfer register. There are known limitations in
some historical systems that would prevent larger blocks from
being accepted. Historical values were chosen to improve
compatibility with historical scripts using dd or similar
utilities to manipulate archives. Also, default block sizes for
any file type other than character special file has been deleted
from this volume of POSIX.1‐2017 as unimportant and not likely to
affect the structure of the resulting archive.
Implementations are permitted to modify the block-size value
based on the archive format or the device to which the archive is
being written. This is to provide implementations with the
opportunity to take advantage of special types of devices, and it
should not be used without a great deal of consideration as it
almost certainly decreases archive portability.
The intended use of the -n option was to permit extraction of one
or more files from the archive without processing the entire
archive. This was viewed by the standard developers as offering
significant performance advantages over historical
implementations. The -n option in early proposals had three
effects; the first was to cause special characters in patterns to
not be treated specially. The second was to cause only the first
file that matched a pattern to be extracted. The third was to
cause pax to write a diagnostic message to standard error when no
file was found matching a specified pattern. Only the second
behavior is retained by this volume of POSIX.1‐2017, for many
reasons. First, it is in general not acceptable for a single
option to have multiple effects. Second, the ability to make
pattern matching characters act as normal characters is useful
for parts of pax other than file extraction. Third, a finer
degree of control over the special characters is useful because
users may wish to normalize only a single special character in a
single filename. Fourth, given a more general escape mechanism,
the previous behavior of the -n option can be easily obtained
using the -s option or a sed script. Finally, writing a
diagnostic message when a pattern specified by the user is
unmatched by any file is useful behavior in all cases.
In this version, the -n was removed from the copy mode synopsis
of pax; it is inapplicable because there are no pattern operands
specified in this mode.
There is another method than pax for copying subtrees in
POSIX.1‐2008 described as part of the cp utility. Both methods
are historical practice: cp provides a simpler, more intuitive
interface, while pax offers a finer granularity of control. Each
provides additional functionality to the other; in particular,
pax maintains the hard-link structure of the hierarchy while cp
does not. It is the intention of the standard developers that the
results be similar (using appropriate option combinations in both
utilities). The results are not required to be identical; there
seemed insufficient gain to applications to balance the
difficulty of implementations having to guarantee that the
results would be exactly identical.
A single archive may span more than one file. It is suggested
that implementations provide informative messages to the user on
standard error whenever the archive file is changed.
The -d option (do not create intermediate directories not listed
in the archive) found in early proposals was originally provided
as a complement to the historic -d option of cpio. It has been
deleted.
The -s option in early proposals specified a subset of the
substitution command from the ed utility. As there was no reason
for only a subset to be supported, the -s option is now
compatible with the current ed specification. Since the delimiter
can be any non-null character, the following usage with single
<space> characters is valid:
pax -s " foo bar " ...
The -t description is worded so as to note that this may cause
the access time update caused by some other activity (which
occurs while the file is being read) to be overwritten.
The default behavior of pax with regard to file modification
times is the same as historical implementations of tar. It is
not the historical behavior of cpio.
Because the -i option uses /dev/tty, utilities without a
controlling terminal are not able to use this option.
The -y option, found in early proposals, has been deleted because
a line containing a single <period> for the -i option has
equivalent functionality. The special lines for the -i option (a
single <period> and the empty line) are historical practice in
cpio.
In early drafts, a -echarmap option was included to increase
portability of files between systems using different coded
character sets. This option was omitted because it was apparent
that consensus could not be formed for it. In this version, the
use of UTF‐8 should be an adequate substitute.
The ISO POSIX‐2:1993 standard and ISO POSIX‐1 standard
requirements for pax, however, made it very difficult to create a
single archive containing files created using extended characters
provided by different locales. This version adds the hdrcharset
keyword to make it possible to archive files in these cases
without dropping files due to translation errors.
Translating filenames and other attributes from a locale's
encoding to UTF‐8 and then back again can lose information, as
the resulting filename might not be byte-for-byte equivalent to
the original. To avoid this problem, users can specify the -o
hdrcharset=binary option, which will cause the resulting archive
to use binary format for all names and attributes. Such archives
are not portable among hosts that use different native encodings
(e.g., EBCDIC versus ASCII-based encodings), but they will allow
interchange among the vast majority of POSIX file systems in
practical use. Also, the -o hdrcharset=binary option will cause
pax in copy mode to behave more like other standard utilities
such as cp.
If the values specified by the -o exthdr.name=value, -o
globexthdr.name=value, or by $TMPDIR (if -o globexthdr.name is
not specified) require a character encoding other than that
described in the ISO/IEC 646:1991 standard, a path extended
header record will have to be created for the file. If a
hdrcharset extended header record is active for such headers, it
will determine the codeset used for the value field in these
extended path header records. These path extended header records
always need to be created when writing an archive even if
hdrcharset=binary has been specified and would contain the same
(binary) data that appears in the ustar header record prefix and
name fields. (In other words, an extended header path record is
always required to be generated if the prefix or name fields
contain non-ASCII characters even when hdrcharset=binary is also
in effect for that file.)
The -k option was added to address international concerns about
the dangers involved in the character set transformations of -e
(if the target character set were different from the source, the
filenames might be transformed into names matching existing
files) and also was made more general to protect files
transferred between file systems with different {NAME_MAX} values
(truncating a filename on a smaller system might also
inadvertently overwrite existing files). As stated, it prevents
any overwriting, even if the target file is older than the
source. This version adds more granularity of options to solve
this problem by introducing the -oinvalid=option—specifically the
UTF‐8 and binary actions. (Note that an existing file is still
subject to overwriting in this case. The -k option closes that
loophole.)
Some of the file characteristics referenced in this volume of
POSIX.1‐2017 might not be supported by some archive formats. For
example, neither the tar nor cpio formats contain the file access
time. For this reason, the e specification character has been
provided, intended to cause all file characteristics specified in
the archive to be retained.
It is required that extracted directories, by default, have their
access and modification times and permissions set to the values
specified in the archive. This has obvious problems in that the
directories are almost certainly modified after being extracted
and that directory permissions may not permit file creation. One
possible solution is to create directories with the mode
specified in the archive, as modified by the umask of the user,
with sufficient permissions to allow file creation. After all
files have been extracted, pax would then reset the access and
modification times and permissions as necessary.
The list-mode formatting description borrows heavily from the one
defined by the printf utility. However, since there is no
separate operand list to get conversion arguments, the format was
extended to allow specifying the name of the conversion argument
as part of the conversion specification.
The T conversion specifier allows time fields to be displayed in
any of the date formats. Unlike the ls utility, pax does not
adjust the format when the date is less than six months in the
past. This makes parsing the output more predictable.
The D conversion specifier handles the ability to display the
major/minor or file size, as with ls, by using %-8(size)D.
The L conversion specifier handles the ls display for symbolic
links.
Conversion specifiers were added to generate existing known types
used for ls.
pax Interchange Format
The new POSIX data interchange format was developed primarily to
satisfy international concerns that the ustar and cpio formats
did not provide for file, user, and group names encoded in
characters outside a subset of the ISO/IEC 646:1991 standard. The
standard developers realized that this new POSIX data interchange
format should be very extensible because there were other
requirements they foresaw in the near future:
* Support international character encodings and locale
information
* Support security information (ACLs, and so on)
* Support future file types, such as realtime or contiguous
files
* Include data areas for implementation use
* Support systems with words larger than 32 bits and timers
with subsecond granularity
The following were not goals for this format because these are
better handled by separate utilities or are inappropriate for a
portable format:
* Encryption
* Compression
* Data translation between locales and codesets
* inode storage
The format chosen to support the goals is an extension of the
ustar format. Of the two formats previously available, only the
ustar format was selected for extensions because:
* It was easier to extend in an upwards-compatible way. It
offered version flags and header block type fields with room
for future standardization. The cpio format, while possessing
a more flexible file naming methodology, could not be
extended without breaking some theoretical implementation or
using a dummy filename that could be a legitimate filename.
* Industry experience since the original ``tar wars'' fought in
developing the ISO POSIX‐1 standard has clearly been in favor
of the ustar format, which is generally the default output
format selected for pax implementations on new systems.
The new format was designed with one additional goal in mind:
reasonable behavior when an older tar or pax utility happened to
read an archive. Since the POSIX.1‐1990 standard mandated that a
``format-reading utility'' had to treat unrecognized typeflag
values as regular files, this allowed the format to include all
the extended information in a pseudo-regular file that preceded
each real file. An option is given that allows the archive
creator to set up reasonable names for these files on the older
systems. Also, the normative text suggests that reasonable file
access values be used for this ustar header block. Making these
header files inaccessible for convenient reading and deleting
would not be reasonable. File permissions of 600 or 700 are
suggested.
The ustar typeflag field was used to accommodate the additional
functionality of the new format rather than magic or version
because the POSIX.1‐1990 standard (and, by reference, the
previous version of pax), mandated the behavior of the format-
reading utility when it encountered an unknown typeflag, but was
silent about the other two fields.
Early proposals for the first version of this standard contained
a proposed archive format that was based on compatibility with
the standard for tape files (ISO 1001, similar to the format used
historically on many mainframes and minicomputers). This format
was overly complex and required considerable overhead in volume
and header records. Furthermore, the standard developers felt
that it would not be acceptable to the community of POSIX
developers, so it was later changed to be a format more closely
related to historical practice on POSIX systems.
The prefix and name split of pathnames in ustar was replaced by
the single path extended header record for simplicity.
The concept of a global extended header (typeflagg) was
controversial. If this were applied to an archive being recorded
on magnetic tape, a few unreadable blocks at the beginning of the
tape could be a serious problem; a utility attempting to extract
as many files as possible from a damaged archive could lose a
large percentage of file header information in this case.
However, if the archive were on a reliable medium, such as a CD‐
ROM, the global extended header offers considerable potential
size reductions by eliminating redundant information. Thus, the
text warns against using the global method for unreliable media
and provides a method for implanting global information in the
extended header for each file, rather than in the typeflag g
records.
No facility for data translation or filtering on a per-file basis
is included because the standard developers could not invent an
interface that would allow this in an efficient manner. If a
filter, such as encryption or compression, is to be applied to
all the files, it is more efficient to apply the filter to the
entire archive as a single file. The standard developers
considered interfaces that would invoke a shell script for each
file going into or out of the archive, but the system overhead in
this approach was considered to be too high.
One such approach would be to have filter= records that give a
pathname for an executable. When the program is invoked, the file
and archive would be open for standard input/output and all the
header fields would be available as environment variables or
command-line arguments. The standard developers did discuss such
schemes, but they were omitted from POSIX.1‐2008 due to concerns
about excessive overhead. Also, the program itself would need to
be in the archive if it were to be used portably.
There is currently no portable means of identifying the character
set(s) used for a file in the file system. Therefore, pax has not
been given a mechanism to generate charset records automatically.
The only portable means of doing this is for the user to write
the archive using the -ocharset=string command line option. This
assumes that all of the files in the archive use the same
encoding. The ``implementation-defined'' text is included to
allow for a system that can identify the encodings used for each
of its files.
The table of standards that accompanies the charset record
description is acknowledged to be very limited. Only a limited
number of character set standards is reasonable for maximal
interchange. Any character set is, of course, possible by prior
agreement. It was suggested that EBCDIC be listed, but it was
omitted because it is not defined by a formal standard. Formal
standards, and then only those with reasonably large followings,
can be included here, simply as a matter of practicality. The
<value>s represent names of officially registered character sets
in the format required by the ISO 2375:1985 standard.
The normal <comma> or <blank>-separated list rules are not
followed in the case of keyword options to allow ease of argument
parsing for getopts.
Further information on character encodings is in pax Archive
Character Set Encoding/Decoding.
The standard developers have reserved keyword name space for
vendor extensions. It is suggested that the format to be used is:
VENDOR.keyword
where VENDOR is the name of the vendor or organization in all
uppercase letters. It is further suggested that the keyword
following the <period> be named differently than any of the
standard keywords so that it could be used for future
standardization, if appropriate, by omitting the VENDOR prefix.
The <length> field in the extended header record was included to
make it simpler to step through the records, even if a record
contains an unknown format (to a particular pax) with complex
interactions of special characters. It also provides a minor
integrity checkpoint within the records to aid a program
attempting to recover files from a damaged archive.
There are no extended header versions of the devmajor and
devminor fields because the unspecified format ustar header field
should be sufficient. If they are not, vendor-specific extended
keywords (such as VENDOR.devmajor) should be used.
Device and i-number labeling of files was not adopted from cpio;
files are interchanged strictly on a symbolic name basis, as in
ustar.
Just as with the ustar format descriptions, the new format makes
no special arrangements for multi-volume archives. Each of the
pax archive types is assumed to be inside a single POSIX file and
splitting that file over multiple volumes (diskettes, tape
cartridges, and so on), processing their labels, and mounting
each in the proper sequence are considered to be implementation
details that cannot be described portably.
The pax format is intended for interchange, not only for backup
on a single (family of) systems. It is not as densely packed as
might be possible for backup:
* It contains information as coded characters that could be
coded in binary.
* It identifies extended records with name fields that could be
omitted in favor of a fixed-field layout.
* It translates names into a portable character set and
identifies locale-related information, both of which are
probably unnecessary for backup.
The requirements on restoring from an archive are slightly
different from the historical wording, allowing for non-
monolithic privilege to bring forward as much as possible. In
particular, attributes such as ``high performance file'' might be
broadly but not universally granted while set-user-ID or chown()
might be much more restricted. There is no implication in
POSIX.1‐2008 that the security information be honored after it is
restored to the file hierarchy, in spite of what might be
improperly inferred by the silence on that topic. That is a topic
for another standard.
Links are recorded in the fashion described here because a link
can be to any file type. It is desirable in general to be able to
restore part of an archive selectively and restore all of those
files completely. If the data is not associated with each link,
it is not possible to do this. However, the data associated with
a file can be large, and when selective restoration is not
needed, this can be a significant burden. The archive is
structured so that files that have no associated data can always
be restored by the name of any link name of any link, and the
user may choose whether data is recorded with each instance of a
file that contains data. The format permits mixing of both types
of links in a single archive; this can be done for special needs,
and pax is expected to interpret such archives on input properly,
despite the fact that there is no pax option that would force
this mixed case on output. (When -o linkdata is used, the output
must contain the duplicate data, but the implementation is free
to include it or omit it when -o linkdata is not used.)
The time values are included as extended header records for those
implementations needing more than the eleven octal digits allowed
by the ustar format. Portable file timestamps cannot be negative.
If pax encounters a file with a negative timestamp in copy or
write mode, it can reject the file, substitute a non-negative
timestamp, or generate a non-portable timestamp with a leading
'-'. Even though some implementations can support finer file-
time granularities than seconds, the normative text requires
support only for seconds since the Epoch because the ISO POSIX‐1
standard states them that way. The ustar format includes only
mtime; the new format adds atime and ctime for symmetry. The
atime access time restored to the file system will be affected by
the -p a and -p e options. The ctime creation time (actually
inode modification time) is described with appropriate privileges
so that it can be ignored when writing to the file system. POSIX
does not provide a portable means to change file creation time.
Nothing is intended to prevent a non-portable implementation of
pax from restoring the value.
The gid, size, and uid extended header records were included to
allow expansion beyond the sizes specified in the regular tar
header. New file system architectures are emerging that will
exhaust the 12-digit size field. There are probably not many
systems requiring more than 8 digits for user and group IDs, but
the extended header values were included for completeness,
allowing overrides for all of the decimal values in the tar
header.
The standard developers intended to describe the effective
results of pax with regard to file ownerships and permissions;
implementations are not restricted in timing or sequencing the
restoration of such, provided the results are as specified.
Much of the text describing the extended headers refers to use in
``write or copy modes''. The copy mode references are due to the
normative text: ``The effect of the copy shall be as if the
copied files were written to an archive file and then
subsequently extracted ...''. There is certainly no way to test
whether pax is actually generating the extended headers in copy
mode, but the effects must be as if it had.
pax Archive Character Set Encoding/Decoding
There is a need to exchange archives of files between systems of
different native codesets. Filenames, group names, and user names
must be preserved to the fullest extent possible when an archive
is read on the receiving platform. Translation of the contents of
files is not within the scope of the pax utility.
There will also be the need to represent characters that are not
available on the receiving platform. These unsupported characters
cannot be automatically folded to the local set of characters due
to the chance of collisions. This could result in overwriting
previous extracted files from the archive or pre-existing files
on the system.
For these reasons, the codeset used to represent characters
within the extended header records of the pax archive must be
sufficiently rich to handle all commonly used character sets. The
fields requiring translation include, at a minimum, filenames,
user names, group names, and link pathnames. Implementations may
wish to have localized extended keywords that use non-portable
characters.
The standard developers considered the following options:
* The archive creator specifies the well-defined name of the
source codeset. The receiver must then recognize the codeset
name and perform the appropriate translations to the
destination codeset.
* The archive creator includes within the archive the character
mapping table for the source codeset used to encode extended
header records. The receiver must then read the character
mapping table and perform the appropriate translations to the
destination codeset.
* The archive creator translates the extended header records in
the source codeset into a canonical form. The receiver must
then perform the appropriate translations to the destination
codeset.
The approach that incorporates the name of the source codeset
poses the problem of codeset name registration, and makes the
archive useless to pax archive decoders that do not recognize
that codeset.
Because parts of an archive may be corrupted, the standard
developers felt that including the character map of the source
codeset was too fragile. The loss of this one key component could
result in making the entire archive useless. (The difference
between this and the global extended header decision was that the
latter has a workaround—duplicating extended header records on
unreliable media—but this would be too burdensome for large
character set maps.)
Both of the above approaches also put an undue burden on the pax
archive receiver to handle the cross-product of all source and
destination codesets.
To simplify the translation from the source codeset to the
canonical form and from the canonical form to the destination
codeset, the standard developers decided that the internal
representation should be a stateless encoding. A stateless
encoding is one where each codepoint has the same meaning,
without regard to the decoder being in a specific state. An
example of a stateful encoding would be the Japanese Shift-JIS;
an example of a stateless encoding would be the ISO/IEC 646:1991
standard (equivalent to 7-bit ASCII).
For these reasons, the standard developers decided to adopt a
canonical format for the representation of file information
strings. The obvious, well-endorsed candidate is the
ISO/IEC 10646‐1:2000 standard (based in part on Unicode), which
can be used to represent the characters of virtually all
standardized character sets. The standard developers initially
agreed upon using UCS2 (16-bit Unicode) as the internal
representation. This repertoire of characters provides a
sufficiently rich set to represent all commonly-used codesets.
However, the standard developers found that the 16-bit Unicode
representation had some problems. It forced the issue of
standardizing byte ordering. The 2-byte length of each character
made the extended header records twice as long for the case of
strings coded entirely from historical 7-bit ASCII. For these
reasons, the standard developers chose the UTF‐8 defined in the
ISO/IEC 10646‐1:2000 standard. This multi-byte representation
encodes UCS2 or UCS4 characters reliably and deterministically,
eliminating the need for a canonical byte ordering. In addition,
NUL octets and other characters possibly confusing to POSIX file
systems do not appear, except to represent themselves. It was
realized that certain national codesets take up more space after
the encoding, due to their placement within the UCS range; it was
felt that the usefulness of the encoding of the names outweighs
the disadvantage of size increase for file, user, and group
names.
The encoding of UTF‐8 is as follows:
UCS4 Hex Encoding UTF-8 Binary Encoding
00000000-0000007F 0xxxxxxx
00000080-000007FF 110xxxxx 10xxxxxx
00000800-0000FFFF 1110xxxx 10xxxxxx 10xxxxxx
00010000-001FFFFF 11110xxx 10xxxxxx 10xxxxxx 10xxxxxx
00200000-03FFFFFF 111110xx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
04000000-7FFFFFFF 1111110x 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx 10xxxxxx
where each 'x' represents a bit value from the character being
translated.
ustar Interchange Format
The description of the ustar format reflects numerous
enhancements over pre-1988 versions of the historical tar
utility. The goal of these changes was not only to provide the
functional enhancements desired, but also to retain compatibility
between new and old versions. This compatibility has been
retained. Archives written using the old archive format are
compatible with the new format.
Implementors should be aware that the previous file format did
not include a mechanism to archive directory type files. For this
reason, the convention of using a filename ending with <slash>
was adopted to specify a directory on the archive.
The total size of the name and prefix fields have been set to
meet the minimum requirements for {PATH_MAX}. If a pathname will
fit within the name field, it is recommended that the pathname be
stored there without the use of the prefix field. Although the
name field is known to be too small to contain {PATH_MAX}
characters, the value was not changed in this version of the
archive file format to retain backwards-compatibility, and
instead the prefix was introduced. Also, because of the earlier
version of the format, there is no way to remove the restriction
on the linkname field being limited in size to just that of the
name field.
The size field is required to be meaningful in all implementation
extensions, although it could be zero. This is required so that
the data blocks can always be properly counted.
It is suggested that if device special files need to be
represented that cannot be represented in the standard format,
that one of the extension types (A‐Z) be used, and that the
additional information for the special file be represented as
data and be reflected in the size field.
Attempting to restore a special file type, where it is converted
to ordinary data and conflicts with an existing filename, need
not be specially detected by the utility. If run as an ordinary
user, pax should not be able to overwrite the entries in, for
example, /dev in any case (whether the file is converted to
another type or not). If run as a privileged user, it should be
able to do so, and it would be considered a bug if it did not.
The same is true of ordinary data files and similarly named
special files; it is impossible to anticipate the needs of the
user (who could really intend to overwrite the file), so the
behavior should be predictable (and thus regular) and rely on the
protection system as required.
The value 7 in the typeflag field is intended to define how
contiguous files can be stored in a ustar archive. POSIX.1‐2008
does not require the contiguous file extension, but does define a
standard way of archiving such files so that all conforming
systems can interpret these file types in a meaningful and
consistent manner. On a system that does not support extended
file types, the pax utility should do the best it can with the
file and go on to the next.
The file protection modes are those conventionally used by the ls
utility. This is extended beyond the usage in the ISO POSIX‐2
standard to support the ``shared text'' or ``sticky'' bit. It is
intended that the conformance document should not document
anything beyond the existence of and support of such a mode.
Further extensions are expected to these bits, particularly with
overloading the set-user-ID and set-group-ID flags.
cpio Interchange Format
The reference to appropriate privileges in the cpio format refers
to an error on standard output; the ustar format does not make
comparable statements.
The model for this format was the historical System V cpio-c data
interchange format. This model documents the portable version of
the cpio format and not the binary version. It has the
flexibility to transfer data of any type described within
POSIX.1‐2008, yet is extensible to transfer data types specific
to extensions beyond POSIX.1‐2008 (for example, contiguous
files). Because it describes existing practice, there is no
question of maintaining upwards-compatibility.
cpio Header
There has been some concern that the size of the c_ino field of
the header is too small to handle those systems that have very
large inode numbers. However, the c_ino field in the header is
used strictly as a hard-link resolution mechanism for archives.
It is not necessarily the same value as the inode number of the
file in the location from which that file is extracted.
The name c_magic is based on historical usage.
cpio Filename
For most historical implementations of the cpio utility,
{PATH_MAX} octets can be used to describe the pathname without
the addition of any other header fields (the NUL character would
be included in this count). {PATH_MAX} is the minimum value for
pathname size, documented as 256 bytes. However, an
implementation may use c_namesize to determine the exact length
of the pathname. With the current description of the <cpio.h>
header, this pathname size can be as large as a number that is
described in six octal digits.
Two values are documented under the c_mode field values to
provide for extensibility for known file types:
0110 000 Reserved for contiguous files. The implementation may
treat the rest of the information for this archive like
a regular file. If this file type is undefined, the
implementation may create the file as a regular file.
This provides for extensibility of the cpio format while allowing
for the ability to read old archives. Files of an unknown type
may be read as ``regular files'' on some implementations. On a
system that does not support extended file types, the pax utility
should do the best it can with the file and go on to the next.
None.
Chapter 2, Shell Command Language, cp(1p), ed(1p), getopts(1p),
ls(1p), printf(1p)
The Base Definitions volume of POSIX.1‐2017, Section 3.169, File
Mode Bits, Chapter 5, File Format Notation, Chapter 8,
Environment Variables, Section 12.2, Utility Syntax Guidelines,
cpio.h(0p), tar.h(0p)
The System Interfaces volume of POSIX.1‐2017, chown(3p),
creat(3p), fstatat(3p), mkdir(3p), mkfifo(3p), utime(3p),
write(3p)
Portions of this text are reprinted and reproduced in electronic
form from IEEE Std 1003.1-2017, Standard for Information
Technology -- Portable Operating System Interface (POSIX), The
Open Group Base Specifications Issue 7, 2018 Edition, Copyright
(C) 2018 by the Institute of Electrical and Electronics
Engineers, Inc and The Open Group. In the event of any
discrepancy between this version and the original IEEE and The
Open Group Standard, the original IEEE and The Open Group
Standard is the referee document. The original Standard can be
obtained online at http://www.opengroup.org/unix/online.html .
Any typographical or formatting errors that appear in this page
are most likely to have been introduced during the conversion of
the source files to man page format. To report such errors, see
https://www.kernel.org/doc/man-pages/reporting_bugs.html .
IEEE/The Open Group 2017 PAX(1P)
Pages that refer to this page: cpio.h(0p), tar.h(0p), ar(1p), cp(1p), file(1p), find(1p), ln(1p)
|
HTML rendering created 2024-06-26 by Michael Kerrisk, author of The Linux Programming Interface. For details of in-depth Linux/UNIX system programming training courses that I teach, look here. Hosting by jambit GmbH. |
|