aboutsummaryrefslogtreecommitdiff
path: root/contrib
AgeCommit message (Collapse)Author
2023-03-23ext2simg: fix same_file() with symlinksEric Biggers
Fix same_file() to use stat() instead of lstat() when checking the paths, so that symlinks are dereferenced. This is needed to be consistent with how the paths are actually accessed later. Otherwise, not all cases where the input and output file are the same are detected. Also just use the stat() result to check whether the output file exists, instead of using a separate call to access(). Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool") Change-Id: Ie36981f9dbc19494732f518488a75fb92c0f0343 Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: fix error check of io_channel_read_blk64()Eric Biggers
Check the return value of io_channel_read_blk64() correctly, considering that it returns an errcode_t, which can be positive. Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool") Change-Id: Iafc6c0169bc8ac79198f285da0246ff3b841ded8 Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: clean up integer types and check for too-large fsEric Biggers
libsparse assumes 32-bit block numbers. Also, ext2simg might read nearly the entire filesystem into memory. Therefore, make ext2simg use appropriate integer types, and explicitly check for when the filesystem is too large or allocating memory failed. Change-Id: Ic415d0e974dce2b4ff6e7fa9265f6e86d371a274 Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: clean up add_chunk()Eric Biggers
Remove a level of indentation, check a bool in the normal way, and simplify the linked list handling. No change in behavior. Change-Id: I12589a254f155b1c40418458a666b87c7ef5c1cf Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: use a standard flexible arrayEric Biggers
Use a standard flexible array instead of a nonstandard zero-length array. No change in behavior. Change-Id: Ifdce24f5d6e2471634bb785527def3fe8fefc202 Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: use bool where appropriateEric Biggers
For the values that get used as the 'bool' parameters of sparse_file_write(), use 'bool' in ext2simg too. No change in behavior. Change-Id: I05f7d6fd3027eb10231c035f9fdc8e946e2c4c90 Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: fix same_file() to check st_devEric Biggers
File identity is determined by the combination of st_dev and st_ino, not by st_ino alone. This fixes a bug where ext2simg would needlessly make a copy of all the data when the input and output files happened to have the same st_ino. Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool") Change-Id: I94e4bf57d9f91b31e5438768805e9f10bec3411d Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-03-23ext2simg: fix off-by-one errors causing corruptionEric Biggers
The chunk_end parameter to add_chunk() is exclusive, but two callers incorrectly treat it as inclusive: when the maximum chunk length of 'INT32_MAX - 12' bytes is reached, and when a chunk extends to the very end of the filesystem. The result is that the output simg file contains zeroes for the last block of these chunks instead of the correct data. A related bug is that the expanded size of the simg file is set to the filesystem size (in blocks) minus s_first_data_block. On filesystems where s_first_data_block != 0, i.e. 1K blocksize filesystems without bigalloc enabled, this truncates the last block of the filesystem. Fix these bugs by (a) making add_chunk() take the chunk length and passing the correct values, and (b) using the filesystem size properly. Here is a reproducer that shows the last block of the filesystem being truncated (bsize=1024) and being corrupted with zeroes (bsize=4096): for bsize in 1024 4096; do rm -f ext4.img mkfs.ext4 -b $bsize ext4.img 10000 mkdir -p mnt sudo mount -t ext4 -o loop ext4.img mnt sudo cp /dev/urandom mnt/fill sudo umount mnt ext2simg ext4.img ext4.simg simg2img ext4.simg ext4.img.restored cmp ext4.img ext4.img.restored done Fixes: db6f320912cf ("AOSP: android: add the ext2simg tool") Reported-by: Clemens Lang <clemens.lang@bmw.de> Change-Id: I3b64c4fbffa5821b431f29e99b36168617da7563 Signed-off-by: Eric Biggers <ebiggers@google.com>
2023-01-30Merge upstream commit 'b0101535a35c' into aosp/masterEric Biggers
Merge the "maint" branch of upstream e2fsprogs. This is 1.46.6-rc1 plus additional changes from git. Change-Id: Ifec9ff2e1ff81e7d3415870d77039cc2f796e3c4
2023-01-25e2fsdroid: stop disabling address sanitizationEric Biggers
Address sanitization was disabled in e2fsdroid over 5 years ago, due to a bug in libext2fs. However, that bug has long since been fixed by upstream commit 689b7be2da01 ("libext2fs: avoid dereferencing beyond allocated memory in xattr code"). So it should be fine to re-enable address sanitization now. Bug: 68387795 Change-Id: I89a7a1ec1a45d0a2ed76d2e5938dbc127eb267a6
2023-01-04Merge upstream tag 'v1.46.5' into aosp/masterEric Biggers
Upgrade from v1.46.2 to v1.46.5. Test: booted Cuttlefish using ext4 userdata Change-Id: I9d4eee0db74cb80b82778b4c5e5a574093c27392
2022-09-13AOSP: e2fsdroid: static import of libbaseAlessio Balsini
Fix a wrong mixed shared/static library inclusion that has been unveiled by a recent clang upgrade to clang-r450784e: the linker couldn't find the requested object reference and caused the tool to crash. libsnapshot_fuzzer_test was luckily catching this misbehaviour as it was crashing as well when trying to format a loop device as ext4. Bug: 230851331 Test: m && atest libsnapshot_fuzzer_test Signed-off-by: Alessio Balsini <balsini@google.com> Change-Id: I20b7b1d66920eb8f767e49311f913564f14ee30e Fix AOSP commit: 83239ca87da0bbdb088be7f0d048472e837899c2
2022-09-13AOSP: Fix e2fsdroid build with muslColin Cross
The e2fsdroid build fails with musl because config.h is not included before ext2fs.h, which causes HAVE_SYS_TYPES_H not to be defined resulting in a missing definition for dev_t. Include config.h at the top of each .c file, and remove extra config.h include from perms.h. Bug: 190084016 Test: m USE_HOST_MUSL=true fastboot Change-Id: I95b3fff3f10ba85c00ec049811dd6b5d412e5dd2 From AOSP commit: 09c63d5edd35e3ca8366be0d92aad922d8895ac1
2022-05-18e2fsdroid: static import of libbaseAlessio Balsini
Fix a wrong mixed shared/static library inclusion that has been unveiled by a recent clang upgrade to clang-r450784e: the linker couldn't find the requested object reference and caused the tool to crash. libsnapshot_fuzzer_test was luckily catching this misbehaviour as it was crashing as well when trying to format a loop device as ext4. Bug: 230851331 Test: m && atest libsnapshot_fuzzer_test Signed-off-by: Alessio Balsini <balsini@google.com> Change-Id: I20b7b1d66920eb8f767e49311f913564f14ee30e
2022-01-04setup-schroot: install the udev and systemd packages separatelyTheodore Ts'o
On non-Linux Debian ports (e.g., GNU/Hurd and GNU/kFreeBSD) the udev and systemd packages don't exist. So try to install them separately, so they can fail on their own on those platforms. Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-12-21setup-schroot: add some additional packages needed to build debian packagesTheodore Ts'o
On older Debian systems, "apt-get build-dep e2fsprogs" might not bring in all of the packages needed to build in the most recent versions of e2fsprogs. So explicitly try to install some additional packages including dh-exec, udev, systemd, and cron. Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-08-24Fix e2fsdroid build with muslColin Cross
The e2fsdroid build fails with musl because config.h is not included before ext2fs.h, which causes HAVE_SYS_TYPES_H not to be defined resulting in a missing definition for dev_t. Include config.h at the top of each .c file, and remove extra config.h include from perms.h. Bug: 190084016 Test: m USE_HOST_MUSL=true fastboot Change-Id: I95b3fff3f10ba85c00ec049811dd6b5d412e5dd2
2021-08-02hashmap: change return value type of ext2fs_hashmap_add()Zhiqiang Liu
In ext2fs_hashmap_add(), new entry is allocated by calling malloc(). If malloc() return NULL, it will cause a segmentation fault problem. Here, we change return value type of ext2fs_hashmap_add() from void to int. If allocating new entry fails, we will return -1, and the callers should also verify the return value of ext2fs_hashmap_add(). Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com> Signed-off-by: Wu Guanghao <wuguanghao3@huawei.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-07-30setup-schroot: install the acl and libreadline-dev packagesTheodore Ts'o
The acl package is needed to run the m_rootdir_acl test. The libreadline-dev package will drag in the shared library package for libreadline (libreadlineN) so that the developer running debugfs will be able to use line editing. Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-07-29contrib: add setup-schroot command for use on Debian porter boxesTheodore Ts'o
Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-07-27AOSP: Move system_shared_libs into target.bionic clauseColin Cross
Use target.bionic.system_shared_libs when it is used to limit the default shared libraries (as opposed to remove them completely). This avoids attempting to add a host dependency on libc when system_shared_libs is modified to apply to all variants. Also remove system_shared_libs from static binaries where it has no effect, and consolidate it into e2fsprogs-defaults. Bug: 193559105 Test: m checkbuild Change-Id: I2d447b006afc783f4acd6c1acd93f338a68a01ed From AOSP commit: 48fa7248112701c30d3cabfb8d3360b2408d6491
2021-07-27AOSP: [LSC] Add LOCAL_LICENSE_KINDS to external/e2fsprogsBob Badour
Added SPDX-license-identifier-0BSD SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.0 SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 SPDX-license-identifier-MIT legacy_notice legacy_unencumbered to: Android.bp Added SPDX-license-identifier-0BSD SPDX-license-identifier-BSD SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 SPDX-license-identifier-MIT legacy_unencumbered to: lib/Android.bp Added SPDX-license-identifier-0BSD SPDX-license-identifier-BSD SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-MIT to: lib/et/Android.bp Added SPDX-license-identifier-0BSD SPDX-license-identifier-MIT to: lib/ss/Android.bp Added SPDX-license-identifier-Apache-2.0 to: contrib/android/Android.bp Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-GPL to: contrib/Android.bp Added SPDX-license-identifier-BSD to: lib/uuid/Android.bp Added SPDX-license-identifier-GPL to: resize/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 to: debugfs/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL to: e2fsck/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 legacy_unencumbered to: lib/ext2fs/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-LGPL to: lib/e2p/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 to: lib/blkid/Android.bp misc/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-MIT to: lib/support/Android.bp Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Exempt-From-Owner-Approval: janitorial work Change-Id: I239a04a83f12ba051be911d18f6df4ae77fb3368 From AOSP commit: e86522c572b5715b85889cf8ca1c52a5cc350ca7
2021-07-21AOSP: e2fsdroid: Don't over-reserve blocks for files in Base FS.David Anderson
If a large number of blocks move from one file to another file, e2fsdroid will inadvertently reserve the moved blocks for the source file. If the larger file is visited first, it can fail to acquire blocks because they're reserved by the smaller file. This patch only reserves the first N blocks of a file in Base FS, where N is large enough to satisfy the |st_size| property from lstat(2). We only consider "owned" blocks rather than deduplicated blocks, in case the new file cannot be deduplicated. Google-Bug-Id: 145316683 Test: e2fsdroid with dynamic partitions Change-Id: I32e255a19550d52d90342c21d7218981108a71b1 From AOSP commit: 2985b26cdbe674084fa02b211503eef9bf970023
2021-07-15append_pathname: check the value returned by reallocwuguanghao
In append_pathname(), we need to add a new path to save the value returned by realloc, otherwise the name->path may be NULL, causing a segfault. Signed-off-by: Wu Guanghao <wuguanghao3@huawei.com> Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-07-13Move system_shared_libs into target.bionic clauseColin Cross
Use target.bionic.system_shared_libs when it is used to limit the default shared libraries (as opposed to remove them completely). This avoids attempting to add a host dependency on libc when system_shared_libs is modified to apply to all variants. Also remove system_shared_libs from static binaries where it has no effect, and consolidate it into e2fsprogs-defaults. Bug: 193559105 Test: m checkbuild Change-Id: I2d447b006afc783f4acd6c1acd93f338a68a01ed
2021-06-15Merge upstream tag 'v1.46.2' into aosp/masterEric Biggers
Conflicts: contrib/android/basefs_allocator.c e2fsck/e2fsck.h e2fsck/pass1.c e2fsck/pass2.c e2fsck/problem.c e2fsck/problem.h lib/e2p/feature.c lib/ext2fs/ext2_fs.h lib/ext2fs/ext2fs.h misc/mke2fs.c misc/tune2fs.c resize/Android.bp Test: booted Cuttlefish using ext4 userdata Change-Id: Iec94e79289afddd7efbca61b076c094da5e1fd6f
2021-02-22misc: replace remaining loff_t with ext2_loff_tAndreas Dilger
Replace the remaining loff_t uses with ext2_loff_t, as was done in patch 1df6a4555, since loff_t is a GCC'ism and is not portable. Signed-off-by: Andreas Dilger <adilger@whamcloud.com> Signed-off-by: Theodore Ts'o <tytso@mit.edu>
2021-02-12[LSC] Add LOCAL_LICENSE_KINDS to external/e2fsprogsBob Badour
Added SPDX-license-identifier-0BSD SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-BSD SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.0 SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 SPDX-license-identifier-MIT legacy_notice legacy_unencumbered to: Android.bp Added SPDX-license-identifier-0BSD SPDX-license-identifier-BSD SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 SPDX-license-identifier-MIT legacy_unencumbered to: lib/Android.bp Added SPDX-license-identifier-0BSD SPDX-license-identifier-BSD SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-MIT to: lib/et/Android.bp Added SPDX-license-identifier-0BSD SPDX-license-identifier-MIT to: lib/ss/Android.bp Added SPDX-license-identifier-Apache-2.0 to: contrib/android/Android.bp Added SPDX-license-identifier-Apache-2.0 SPDX-license-identifier-GPL to: contrib/Android.bp Added SPDX-license-identifier-BSD to: lib/uuid/Android.bp Added SPDX-license-identifier-GPL to: resize/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 to: debugfs/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL to: e2fsck/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-GPL-2.0 SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 legacy_unencumbered to: lib/ext2fs/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-LGPL to: lib/e2p/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-LGPL SPDX-license-identifier-LGPL-2.1 SPDX-license-identifier-LGPL-3.0 to: lib/blkid/Android.bp misc/Android.bp Added SPDX-license-identifier-GPL SPDX-license-identifier-MIT to: lib/support/Android.bp Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Exempt-From-Owner-Approval: janitorial work Change-Id: I239a04a83f12ba051be911d18f6df4ae77fb3368
2021-01-19Merge branch 'maint' into nextTheodore Ts'o
2020-10-01AOSP: Fix a trivial type errorYi Kong
Comparing unsigned int with ULONG_MAX is always false. Signed-off-by: Yi Kong <yikong@google.com> Change-Id: Iae02aad1bcb271d3468828977be288ad04333821 From AOSP commit: 757a4d672dae1a15c57f5f0705ba90ed007da7e6
2020-10-01AOSP: Include private/fs_config.h directly when neededTom Cherry
This is no longer a transitive include of android_filesystem_config.h Bug: 149785767 Test: build Change-Id: I954adf7879fa811eb2b290a0983c84d47ecae26c From AOSP commit: 9fa6bed983d5ddd7226af647a0c4c0d922227be4
2020-07-24Fix a trivial type errorYi Kong
Comparing unsigned int with ULONG_MAX is always false. Signed-off-by: Yi Kong <yikong@google.com> Change-Id: Iae02aad1bcb271d3468828977be288ad04333821
2020-03-21Merge tag 'v1.45.6' into nextTheodore Ts'o
v1.45.6
2020-03-20AOSP: e2fsdroid: Don't skip unusable blocks in BaseFS.David Anderson
Currently, basefs_allocator will iterate through blocks owned by an inode until it finds a block that is free. This effectively ignores the logical to physical block mapping, which can lead to a bigger delta in the final image. An example of how this can happen is if the BaseFS has a deduplicated block (D), that is not deduplicated in the new image: Old image: 1 2 3 D 4 5 New image: 1 2 3 ? 4 5 If the allocator sees that "D" is not usable, and skips to block "4", we will have a non-ideal assignment. Bad image: 1 2 3 4 5 ? This patch refactors get_next_block() to acquire at most one block. It's called a single time, and then only called in a loop if absolutely no blocks can be acquired from anywhere else. In a Virtual A/B simulation, this reduces the COW snapshot size by about 90MB. Bug: 139201772 Test: manual test Change-Id: I354f0dee1ee191dba0e1f90491ed591dba388f7f From AOSP commit: a495b54f89b2ec0e46be8e3564e4852c6434687c
2020-03-20AOSP: e2fsdroid: Fix logical block sequencing in BaseFS.David Anderson
By iterating over blocks to write BaseFS, holes in the extent tree are skipped. This is problematic because the purpose of BaseFS is to preserve the logical to physical block assignment between builds. By not preserving the location of holes, the assignment can be incorrect. For example, consider the following block list for a file: 1 2 3 0 4 5 If this is recorded as: 1 2 3 4 5 If the first block changes to a hole, the intended mapping will not be preserved at all: 0 1 2 0 3 This patch makes two changes to e2fsdroid to fix this. The first change is that holes are now recorded in BaseFS, by iterating over the extent tree rather than the block list, and inserting zeroes where appropriate. The second change is that the block allocator now recognizes when blocks have been skipped (either to deduplication or to holes), and skips the same number of logical blocks in BaseFS as well. In a Virtual A/B simulation, this reduces the COW snapshot size by approximately 100MB. Bug: 139201772 Test: m target-files-package, inspect .map files From AOSP commit: d391f3bf38cbe51718d5c3c0bb3e72b1a9978625
2020-03-20AOSP: e2fsdroid: Properly free the dedup block map.David Anderson
When BaseFS specifies the same block for two files, it gets added to a separate "dedup" bitmap, and removed from the free block bitmap. If the new build does not use every block in this bitmap, there will be an inconsistency: the block bitmap marks blocks as in-use when they are actually free. Although this doesn't matter for AOSP's read-only file systems, it does cause e2fsck to complain, which breaks the build. Fix the inconsistency by properly freeing all unused blocks within the dedup block set. Bug: 139201772 Test: build AOSP using BaseFS Change-Id: I6b6511eb713a56fec932f1d5668f1766d64d9479 From AOSP commit: 346bee6f8b97aefe7714688f738606c116099fbc
2020-03-05Include private/fs_config.h directly when neededTom Cherry
This is no longer a transitive include of android_filesystem_config.h Bug: 149785767 Test: build Change-Id: I954adf7879fa811eb2b290a0983c84d47ecae26c
2020-02-15e2fsdroid: Don't skip unusable blocks in BaseFS.David Anderson
Currently, basefs_allocator will iterate through blocks owned by an inode until it finds a block that is free. This effectively ignores the logical to physical block mapping, which can lead to a bigger delta in the final image. An example of how this can happen is if the BaseFS has a deduplicated block (D), that is not deduplicated in the new image: Old image: 1 2 3 D 4 5 New image: 1 2 3 ? 4 5 If the allocator sees that "D" is not usable, and skips to block "4", we will have a non-ideal assignment. Bad image: 1 2 3 4 5 ? This patch refactors get_next_block() to acquire at most one block. It's called a single time, and then only called in a loop if absolutely no blocks can be acquired from anywhere else. In a Virtual A/B simulation, this reduces the COW snapshot size by about 90MB. Bug: 139201772 Test: manual test Change-Id: I354f0dee1ee191dba0e1f90491ed591dba388f7f
2020-02-15e2fsdroid: Fix logical block sequencing in BaseFS.David Anderson
By iterating over blocks to write BaseFS, holes in the extent tree are skipped. This is problematic because the purpose of BaseFS is to preserve the logical to physical block assignment between builds. By not preserving the location of holes, the assignment can be incorrect. For example, consider the following block list for a file: 1 2 3 0 4 5 If this is recorded as: 1 2 3 4 5 If the first block changes to a hole, the intended mapping will not be preserved at all: 0 1 2 0 3 This patch makes two changes to e2fsdroid to fix this. The first change is that holes are now recorded in BaseFS, by iterating over the extent tree rather than the block list, and inserting zeroes where appropriate. The second change is that the block allocator now recognizes when blocks have been skipped (either to deduplication or to holes), and skips the same number of logical blocks in BaseFS as well. In a Virtual A/B simulation, this reduces the COW snapshot size by approximately 100MB. Bug: 139201772 Test: m target-files-package, inspect .map files
2020-02-07e2fsdroid: Properly free the dedup block map.David Anderson
When BaseFS specifies the same block for two files, it gets added to a separate "dedup" bitmap, and removed from the free block bitmap. If the new build does not use every block in this bitmap, there will be an inconsistency: the block bitmap marks blocks as in-use when they are actually free. Although this doesn't matter for AOSP's read-only file systems, it does cause e2fsck to complain, which breaks the build. Fix the inconsistency by properly freeing all unused blocks within the dedup block set. Bug: 139201772 Test: build AOSP using BaseFS Change-Id: I6b6511eb713a56fec932f1d5668f1766d64d9479
2020-01-02Merge branch 'maint' into nextTheodore Ts'o
2020-01-01AOSP: e2fsdroid: Skip Base FS entries that no longer exist.David Anderson
Don't reserve blocks in the base map if the file does not exist in the target image. This can happen if a file is removed or renamed in between two builds. If the removed file is quite large, skipping it is important since otherwise it will prevent blocks from being allocated for new files. Bug: 145316683 Test: e2fsdroid with dynamic partitions Change-Id: I63a9372c58adeaae3e1235fd92fed78a284ed391 From AOSP commit: 2b6646a3d2a7dd9972275cb829239ae033762da5
2020-01-01AOSP: e2fsdroid: Allow re-use of deduplicated blocks.David Anderson
When using a Base FS map, track deduplicated blocks in a separate bitmap. The first inode to request a block from this set will succeed in getting the block. Blocks in the dedup set are not available for libext2fs to allocate; this ensures that previously deduplicated blocks are re-used for their original purpose. Note that deduplication takes priority over block allocation, so that once a block is removed from the dedup set, that does not actually prevent it from being re-used. Similarly, a file that was not previously sharing a block may have its blocks shared in the new image. Bug: 145316683 Test: e2fsdroid with dynamic partitions Change-Id: I73856faa5d294a7b5fb985ccd1a6974a989481ea From AOSP commit: 4e55425ff5d7d7cea27dcf79125766762e2b3529
2020-01-01AOSP: e2fsdroid: Do not allocate invalid blocks from BaseFS.David Anderson
If certain metadata properties change in between builds, such as the inode table size, then block mappings may not be valid from one build to the next. For example, build A could allocate block N for a file. If build B has a larger inode table, block N may no longer be a data block. In this case, we need to remove the block from the BaseFS range list so we do not give it back to libext2fs for writing data. We do this with a new "owned_block_map" bitmap. If a block from the base FS is not in use by the initial image, and is not used by any other file, it is considered owned, and can be claimed by that file. Note that while this produces correct images, it also prevents deduplicated blocks from being re-used. This will be addressed in a follow-up patch. Bug: 145316683 Test: e2fsdroid block mapping with dynamic partitions Change-Id: I3145e45156f7879bdf956384723fab4bd69acb93 Signed-off-by: David Anderson <dvander@google.com> From AOSP commit: b3d1ccdb673772588b7bb14c4581980ef549a0b8
2020-01-01AOSP: e2fsdroid: Refactor basefs allocation.David Anderson
This refactors base_fs_alloc_load() to address two issues. First, the failure cases have been made simpler by factoring out a common helper for freeing a base_fs_allocator. Second, we no longer return EXIT_FAILURE, since this is not an errcode_t. Bug: 145316683 Change-Id: I8b28c684ecf3423b605b550177d7ead4eb2821a4 From AOSP commit: b461ade872a809ab913d7506e9b8ed597d57dc87
2020-01-01AOSP: e2fsdroid: Refactor block_range.David Anderson
block_range is a singly-linked list, but the head/tail links are manually managed all over. Instead, introduce a block_range_list structure and refactor list helpers to operate on this instead. This ensures head/tail are maintained properly (in some cases, like delete_block_range, they were not). Bug: 145316683 Test: manual test Change-Id: Ieec6324549e2c3a71129871f703f4f0a37aeb1fa From AOSP commit: 4220594792297619d2e70a29476667d1698dbd63
2019-12-10e2fsdroid: Don't over-reserve blocks for files in Base FS.David Anderson
If a large number of blocks move from one file to another file, e2fsdroid will inadvertently reserve the moved blocks for the source file. If the larger file is visited first, it can fail to acquire blocks because they're reserved by the smaller file. This patch only reserves the first N blocks of a file in Base FS, where N is large enough to satisfy the |st_size| property from lstat(2). We only consider "owned" blocks rather than deduplicated blocks, in case the new file cannot be deduplicated. Bug: 145316683 Test: e2fsdroid with dynamic partitions Change-Id: I32e255a19550d52d90342c21d7218981108a71b1
2019-12-09e2fsdroid: Skip Base FS entries that no longer exist.David Anderson
Don't reserve blocks in the base map if the file does not exist in the target image. This can happen if a file is removed or renamed in between two builds. If the removed file is quite large, skipping it is important since otherwise it will prevent blocks from being allocated for new files. Bug: 145316683 Test: e2fsdroid with dynamic partitions Change-Id: I63a9372c58adeaae3e1235fd92fed78a284ed391
2019-12-09e2fsdroid: Allow re-use of deduplicated blocks.David Anderson
When using a Base FS map, track deduplicated blocks in a separate bitmap. The first inode to request a block from this set will succeed in getting the block. Blocks in the dedup set are not available for libext2fs to allocate; this ensures that previously deduplicated blocks are re-used for their original purpose. Note that deduplication takes priority over block allocation, so that once a block is removed from the dedup set, that does not actually prevent it from being re-used. Similarly, a file that was not previously sharing a block may have its blocks shared in the new image. Bug: 145316683 Test: e2fsdroid with dynamic partitions Change-Id: I73856faa5d294a7b5fb985ccd1a6974a989481ea
2019-12-09e2fsdroid: Do not allocate invalid blocks from BaseFS.David Anderson
If certain metadata properties change in between builds, such as the inode table size, then block mappings may not be valid from one build to the next. For example, build A could allocate block N for a file. If build B has a larger inode table, block N may no longer be a data block. In this case, we need to remove the block from the BaseFS range list so we do not give it back to libext2fs for writing data. We do this with a new "owned_block_map" bitmap. If a block from the base FS is not in use by the initial image, and is not used by any other file, it is considered owned, and can be claimed by that file. Note that while this produces correct images, it also prevents deduplicated blocks from being re-used. This will be addressed in a follow-up patch. Bug: 145316683 Test: e2fsdroid block mapping with dynamic partitions Change-Id: I3145e45156f7879bdf956384723fab4bd69acb93 Signed-off-by: David Anderson <dvander@google.com>
generated by cgit v1.2.3 (git 2.25.1) at 2024-06-21 01:55:23 +0000