diff --git a/NEWS b/NEWS
index 4b5a523a..03cbdc92 100644
--- a/NEWS
+++ b/NEWS
@@ -1,1477 +1,1480 @@
0.16.4 UNRELEASED
TEST FIXES
* Skip test that requires sync to synchronize filesystems if os.sync is
not available. (Koen Martens)
IMPROVEMENTS
* Implement MemoryRepo.{set_description,get_description}.
(Jelmer Vernooij)
* Raise exception in Repo.stage() when absolute paths are
passed in. Allow passing in relative paths to
porcelain.add().(Jelmer Vernooij)
BUG FIXES
* Handle multi-line quoted values in config files.
(Jelmer Vernooij, #495)
* Allow porcelain.clone of repository without HEAD.
(Jelmer Vernooij, #501)
+ * Support passing tag ids to Walker()'s include argument.
+ (Jelmer Vernooij)
+
0.16.3 2016-01-14
TEST FIXES
* Remove racy check that relies on clock time changing between writes.
(Jelmer Vernooij)
IMPROVEMENTS
* Add porcelain.remote_add. (Jelmer Vernooij)
0.16.2 2016-01-14
IMPROVEMENTS
* Fixed failing test-cases on windows.
(Koen Martens)
API CHANGES
* Repo is now a context manager, so that it can be easily
closed using a ``with`` statement. (Søren Løvborg)
TEST FIXES
* Only run worktree list compat tests against git 2.7.0,
when 'git worktree list' was introduced. (Jelmer Vernooij)
BUG FIXES
* Ignore filemode when building index when core.filemode
is false.
(Koen Martens)
* Initialize core.filemode configuration setting by
probing the filesystem for trustable permissions.
(Koen Martens)
* Fix ``porcelain.reset`` to respect the comittish argument.
(Koen Martens)
* Fix dulwich.porcelain.ls_remote() on Python 3.
(#471, Jelmer Vernooij)
* Allow both unicode and byte strings for host paths
in dulwich.client. (#435, Jelmer Vernooij)
* Add remote from porcelain.clone. (#466, Jelmer Vernooij)
* Fix unquoting of credentials before passing to urllib2.
(#475, Volodymyr Holovko)
* Cope with submodules in `build_index_from_tree`.
(#477, Jelmer Vernooij)
* Handle deleted files in `get_unstaged_changes`.
(#483, Doug Hellmann)
* Don't overwrite files when they haven't changed in
`build_file_from_blob`.
(#479, Benoît HERVIER)
* Check for existence of index file before opening pack.
Fixes a race when new packs are being added.
(#482, wme)
0.16.1 2016-12-25
BUG FIXES
* Fix python3 compatibility for dulwich.contrib.release_robot.
(Jelmer Vernooij)
0.16.0 2016-12-24
IMPROVEMENTS
* Add support for worktrees. See `git-worktree(1)` and
`gitrepository-layout(5)`. (Laurent Rineau)
* Add support for `commondir` file in Git control
directories. (Laurent Rineau)
* Add support for passwords in HTTP URLs.
(Jon Bain, Mika Mäenpää)
* Add `release_robot` script to contrib,
allowing easy finding of current version based on Git tags.
(Mark Mikofski)
* Add ``Blob.splitlines`` method.
(Jelmer Vernooij)
BUG FIXES
* Fix handling of ``Commit.tree`` being set to an actual
tree object rather than a tree id. (Jelmer Vernooij)
* Return remote refs from LocalGitClient.fetch_pack(),
consistent with the documentation for that method.
(#461, Jelmer Vernooij)
* Fix handling of unknown URL schemes in get_transport_and_path.
(#465, Jelmer Vernooij)
0.15.0 2016-10-09
BUG FIXES
* Allow missing trailing LF when reading service name from
HTTP servers. (Jelmer Vernooij, Andrew Shadura, #442)
* Fix dulwich.porcelain.pull() on Python3. (Jelmer Vernooij, #451)
* Properly pull in tags during dulwich.porcelain.clone.
(Jelmer Vernooij, #408)
CHANGES
* Changed license from "GNU General Public License, version 2.0 or later"
to "Apache License, version 2.0 or later or GNU General Public License,
version 2.0 or later". (#153)
IMPROVEMENTS
* Add ``dulwich.porcelain.ls_tree`` implementation. (Jelmer Vernooij)
0.14.1 2016-07-05
BUG FIXES
* Fix regression removing untouched refs when pushing over SSH.
(Jelmer Vernooij #441)
* Skip Python3 tests for SWIFT contrib module, as it has not yet
been ported.
0.14.0 2016-07-03
BUG FIXES
* Fix ShaFile.id after modification of a copied ShaFile.
(Félix Mattrat, Jelmer Vernooij)
* Support removing refs from porcelain.push.
(Jelmer Vernooij, #437)
* Stop magic protocol ref `capabilities^{}` from leaking out
to clients. (Jelmer Vernooij, #254)
IMPROVEMENTS
* Add `dulwich.config.parse_submodules` function.
* Add `RefsContainer.follow` method. (#438)
0.13.0 2016-04-24
IMPROVEMENTS
* Support `ssh://` URLs in get_transport_and_path_from_url().
(Jelmer Vernooij, #402)
* Support missing empty line after headers in Git commits and tags.
(Nicolas Dandrimont, #413)
* Fix `dulwich.porcelain.status` when used in empty trees.
(Jelmer Vernooij, #415)
* Return copies of objects in MemoryObjectStore rather than
references, making the behaviour more consistent with that of
DiskObjectStore. (Félix Mattrat, Jelmer Vernooij)
* Fix ``dulwich.web`` on Python3. (#295, Jonas Haag)
CHANGES
* Drop support for Python 2.6.
* Fix python3 client web support. (Jelmer Vernooij)
BUG FIXES
* Fix hang on Gzip decompression. (Jonas Haag)
* Don't rely on working tell() and seek() methods
on wsgi.input. (Jonas Haag)
* Support fastexport/fastimport functionality on python3 with newer
versions of fastimport (>= 0.9.5). (Jelmer Vernooij, Félix Mattrat)
0.12.0 2015-12-13
IMPROVEMENTS
* Add a `dulwich.archive` module that can create tarballs.
Based on code from Jonas Haag in klaus.
* Add a `dulwich.reflog` module for reading and writing reflogs.
(Jelmer Vernooij)
* Fix handling of ambiguous refs in `parse_ref` to make
it match the behaviour described in https://git-scm.com/docs/gitrevisions.
(Chris Bunney)
* Support Python3 in C modules. (Lele Gaifax)
BUG FIXES
* Simplify handling of SSH command invocation.
Fixes quoting of paths. Thanks, Thomas Liebetraut. (#384)
* Fix inconsistent handling of trailing slashes for DictRefsContainer. (#383)
* Add hack to support thin packs duing fetch(), albeit while requiring the
entire pack file to be loaded into memory. (jsbain)
CHANGES
* This will be the last release to support Python 2.6.
0.11.2 2015-09-18
IMPROVEMENTS
* Add support for agent= capability. (Jelmer Vernooij, #298)
* Add support for quiet capability. (Jelmer Vernooij)
CHANGES
* The ParamikoSSHVendor class has been moved to
* dulwich.contrib.paramiko_vendor, as it's currently untested.
(Jelmer Vernooij, #364)
0.11.1 2015-09-13
Fix-up release to exclude broken blame.py file.
0.11.0 2015-09-13
IMPROVEMENTS
* Extended Python3 support to most of the codebase.
(Gary van der Merwe, Jelmer Vernooij)
* The `Repo` object has a new `close` method that can be called to close any
open resources. (Gary van der Merwe)
* Support 'git.bat' in SubprocessGitClient on Windows.
(Stefan Zimmermann)
* Advertise 'ofs-delta' capability in receive-pack server side
capabilities. (Jelmer Vernooij)
* Switched `default_local_git_client_cls` to `LocalGitClient`.
(Gary van der Merwe)
* Add `porcelain.ls_remote` and `GitClient.get_refs`.
(Michael Edgar)
* Add `Repo.discover` method. (B. M. Corser)
* Add `dulwich.objectspec.parse_refspec`. (Jelmer Vernooij)
* Add `porcelain.pack_objects` and `porcelain.repack`.
(Jelmer Vernooij)
BUG FIXES
* Fix handling of 'done' in graph walker and implement the
'no-done' capability. (Tommy Yu, #88)
* Avoid recursion limit issues resolving deltas. (William Grant, #81)
* Allow arguments in local client binary path overrides.
(Jelmer Vernooij)
* Fix handling of commands with arguments in paramiko SSH
client. (Andreas Klöckner, Jelmer Vernooij, #363)
* Fix parsing of quoted strings in configs. (Jelmer Vernooij, #305)
0.10.1 2015-03-25
BUG FIXES
* Return `ApplyDeltaError` when encountering delta errors
in both C extensions and native delta application code.
(Jelmer Vernooij, #259)
0.10.0 2015-03-22
BUG FIXES
* In dulwich.index.build_index_from_tree, by default
refuse to create entries that start with .git/.
* Fix running of testsuite when installed.
(Jelmer Vernooij, #223)
* Use a block cache in _find_content_rename_candidates(),
improving performance. (Mike Williams)
* Add support for ``core.protectNTFS`` setting.
(Jelmer Vernooij)
* Fix TypeError when fetching empty updates.
(Hwee Miin Koh)
* Resolve delta refs when pulling into a MemoryRepo.
(Max Shawabkeh, #256)
* Fix handling of tags of non-commits in missing object finder.
(Augie Fackler, #211)
* Explicitly disable mmap on plan9 where it doesn't work.
(Jeff Sickel)
IMPROVEMENTS
* New public method `Repo.reset_index`. (Jelmer Vernooij)
* Prevent duplicate parsing of loose files in objects
directory when reading. Thanks to David Keijser for the
report. (Jelmer Vernooij, #231)
0.9.9 2015-03-20
SECURITY BUG FIXES
* Fix buffer overflow in C implementation of pack apply_delta().
(CVE-2015-0838)
Thanks to Ivan Fratric of the Google Security Team for
reporting this issue.
(Jelmer Vernooij)
0.9.8 2014-11-30
BUG FIXES
* Various fixes to improve test suite running on Windows.
(Gary van der Merwe)
* Limit delta copy length to 64K in v2 pack files. (Robert Brown)
* Strip newline from final ACKed SHA while fetching packs.
(Michael Edgar)
* Remove assignment to PyList_SIZE() that was causing segfaults on
pypy. (Jelmer Vernooij, #196)
IMPROVEMENTS
* Add porcelain 'receive-pack' and 'upload-pack'. (Jelmer Vernooij)
* Handle SIGINT signals in bin/dulwich. (Jelmer Vernooij)
* Add 'status' support to bin/dulwich. (Jelmer Vernooij)
* Add 'branch_create', 'branch_list', 'branch_delete' porcelain.
(Jelmer Vernooij)
* Add 'fetch' porcelain. (Jelmer Vernooij)
* Add 'tag_delete' porcelain. (Jelmer Vernooij)
* Add support for serializing/deserializing 'gpgsig' attributes in Commit.
(Jelmer Vernooij)
CHANGES
* dul-web is now available as 'dulwich web-daemon'.
(Jelmer Vernooij)
* dulwich.porcelain.tag has been renamed to tag_create.
dulwich.porcelain.list_tags has been renamed to tag_list.
(Jelmer Vernooij)
API CHANGES
* Restore support for Python 2.6. (Jelmer Vernooij, Gary van der Merwe)
0.9.7 2014-06-08
BUG FIXES
* Fix tests dependent on hash ordering. (Michael Edgar)
* Support staging symbolic links in Repo.stage.
(Robert Brown)
* Ensure that all files object are closed when running the test suite.
(Gary van der Merwe)
* When writing OFS_DELTA pack entries, write correct offset.
(Augie Fackler)
* Fix handler of larger copy operations in packs. (Augie Fackler)
* Various fixes to improve test suite running on Windows.
(Gary van der Merwe)
* Fix logic for extra adds of identical files in rename detector.
(Robert Brown)
IMPROVEMENTS
* Add porcelain 'status'. (Ryan Faulkner)
* Add porcelain 'daemon'. (Jelmer Vernooij)
* Add `dulwich.greenthreads` module which provides support
for concurrency of some object store operations.
(Fabien Boucher)
* Various changes to improve compatibility with Python 3.
(Gary van der Merwe, Hannu Valtonen, michael-k)
* Add OpenStack Swift backed repository implementation
in dulwich.contrib. See README.swift for details. (Fabien Boucher)
API CHANGES
* An optional close function can be passed to the Protocol class. This will
be called by its close method. (Gary van der Merwe)
* All classes with close methods are now context managers, so that they can
be easily closed using a `with` statement. (Gary van der Merwe)
* Remove deprecated `num_objects` argument to `write_pack` methods.
(Jelmer Vernooij)
OTHER CHANGES
* The 'dul-daemon' script has been removed. The same functionality
is now available as 'dulwich daemon'. (Jelmer Vernooij)
0.9.6 2014-04-23
IMPROVEMENTS
* Add support for recursive add in 'git add'.
(Ryan Faulkner, Jelmer Vernooij)
* Add porcelain 'list_tags'. (Ryan Faulkner)
* Add porcelain 'push'. (Ryan Faulkner)
* Add porcelain 'pull'. (Ryan Faulkner)
* Support 'http.proxy' in HttpGitClient.
(Jelmer Vernooij, #1096030)
* Support 'http.useragent' in HttpGitClient.
(Jelmer Vernooij)
* In server, wait for clients to send empty list of
wants when talking to empty repository.
(Damien Tournoud)
* Various changes to improve compatibility with
Python 3. (Gary van der Merwe)
BUG FIXES
* Support unseekable 'wsgi.input' streams.
(Jonas Haag)
* Raise TypeError when passing unicode() object
to Repo.__getitem__.
(Jonas Haag)
* Fix handling of `reset` command in dulwich.fastexport.
(Jelmer Vernooij, #1249029)
* In client, don't wait for server to close connection
first. Fixes hang when used against GitHub
server implementation. (Siddharth Agarwal)
* DeltaChainIterator: fix a corner case where an object is inflated as an
object already in the repository.
(Damien Tournoud, #135)
* Stop leaking file handles during pack reload. (Damien Tournoud)
* Avoid reopening packs during pack cache reload. (Jelmer Vernooij)
API CHANGES
* Drop support for Python 2.6. (Jelmer Vernooij)
0.9.5 2014-02-23
IMPROVEMENTS
* Add porcelain 'tag'. (Ryan Faulkner)
* New module `dulwich.objectspec` for parsing strings referencing
objects and commit ranges. (Jelmer Vernooij)
* Add shallow branch support. (milki)
* Allow passing urllib2 `opener` into HttpGitClient.
(Dov Feldstern, #909037)
CHANGES
* Drop support for Python 2.4 and 2.5. (Jelmer Vernooij)
API CHANGES
* Remove long deprecated ``Repo.commit``, ``Repo.get_blob``,
``Repo.tree`` and ``Repo.tag``. (Jelmer Vernooij)
* Remove long deprecated ``Repo.revision_history`` and ``Repo.ref``.
(Jelmer Vernooij)
* Remove long deprecated ``Tree.entries``. (Jelmer Vernooij)
BUG FIXES
* Raise KeyError rather than TypeError when passing in
unicode object of length 20 or 40 to Repo.__getitem__.
(Jelmer Vernooij)
* Use 'rm' rather than 'unlink' in tests, since the latter
does not exist on OpenBSD and other platforms.
(Dmitrij D. Czarkoff)
0.9.4 2013-11-30
IMPROVEMENTS
* Add ssh_kwargs attribute to ParamikoSSHVendor. (milki)
* Add Repo.set_description(). (Víðir Valberg Guðmundsson)
* Add a basic `dulwich.porcelain` module. (Jelmer Vernooij, Marcin Kuzminski)
* Various performance improvements for object access.
(Jelmer Vernooij)
* New function `get_transport_and_path_from_url`,
similar to `get_transport_and_path` but only
supports URLs.
(Jelmer Vernooij)
* Add support for file:// URLs in `get_transport_and_path_from_url`.
(Jelmer Vernooij)
* Add LocalGitClient implementation.
(Jelmer Vernooij)
BUG FIXES
* Support filesystems with 64bit inode and device numbers.
(André Roth)
CHANGES
* Ref handling has been moved to dulwich.refs.
(Jelmer Vernooij)
API CHANGES
* Remove long deprecated RefsContainer.set_ref().
(Jelmer Vernooij)
* Repo.ref() is now deprecated in favour of Repo.refs[].
(Jelmer Vernooij)
FEATURES
* Add support for graftpoints. (milki)
0.9.3 2013-09-27
BUG FIXES
* Fix path for stdint.h in MANIFEST.in. (Jelmer Vernooij)
0.9.2 2013-09-26
BUG FIXES
* Include stdint.h in MANIFEST.in (Mark Mikofski)
0.9.1 2013-09-22
BUG FIXES
* Support lookups of 40-character refs in BaseRepo.__getitem__. (Chow Loong Jin, Jelmer Vernooij)
* Fix fetching packs with side-band-64k capability disabled. (David Keijser, Jelmer Vernooij)
* Several fixes in send-pack protocol behaviour - handling of empty pack files and deletes.
(milki, #1063087)
* Fix capability negotiation when fetching packs over HTTP.
(#1072461, William Grant)
* Enforce determine_wants returning an empty list rather than None. (Fabien Boucher, Jelmer Vernooij)
* In the server, support pushes just removing refs. (Fabien Boucher, Jelmer Vernooij)
IMPROVEMENTS
* Support passing a single revision to BaseRepo.get_walker() rather than a list of revisions.
(Alberto Ruiz)
* Add `Repo.get_description` method. (Jelmer Vernooij)
* Support thin packs in Pack.iterobjects() and Pack.get_raw().
(William Grant)
* Add `MemoryObjectStore.add_pack` and `MemoryObjectStore.add_thin_pack` methods.
(David Bennett)
* Add paramiko-based SSH vendor. (Aaron O'Mullan)
* Support running 'dulwich.server' and 'dulwich.web' using 'python -m'.
(Jelmer Vernooij)
* Add ObjectStore.close(). (Jelmer Vernooij)
* Raise appropriate NotImplementedError when encountering dumb HTTP servers.
(Jelmer Vernooij)
API CHANGES
* SSHVendor.connect_ssh has been renamed to SSHVendor.run_command.
(Jelmer Vernooij)
* ObjectStore.add_pack() now returns a 3-tuple. The last element will be an
abort() method that can be used to cancel the pack operation.
(Jelmer Vernooij)
0.9.0 2013-05-31
BUG FIXES
* Push efficiency - report missing objects only. (#562676, Artem Tikhomirov)
* Use indentation consistent with C Git in config files.
(#1031356, Curt Moore, Jelmer Vernooij)
* Recognize and skip binary files in diff function.
(Takeshi Kanemoto)
* Fix handling of relative paths in dulwich.client.get_transport_and_path.
(Brian Visel, #1169368)
* Preserve ordering of entries in configuration.
(Benjamin Pollack)
* Support ~ expansion in SSH client paths. (milki, #1083439)
* Support relative paths in alternate paths.
(milki, Michel Lespinasse, #1175007)
* Log all error messages from wsgiref server to the logging module. This
makes the test suit quiet again. (Gary van der Merwe)
* Support passing None for empty tree in changes_from_tree.
(Kevin Watters)
* Support fetching empty repository in client. (milki, #1060462)
IMPROVEMENTS:
* Add optional honor_filemode flag to build_index_from_tree.
(Mark Mikofski)
* Support core/filemode setting when building trees. (Jelmer Vernooij)
* Add chapter on tags in tutorial. (Ryan Faulkner)
FEATURES
* Add support for mergetags. (milki, #963525)
* Add support for posix shell hooks. (milki)
0.8.7 2012-11-27
BUG FIXES
* Fix use of alternates in ``DiskObjectStore``.{__contains__,__iter__}.
(Dmitriy)
* Fix compatibility with Python 2.4. (David Carr)
0.8.6 2012-11-09
API CHANGES
* dulwich.__init__ no longer imports client, protocol, repo and
server modules. (Jelmer Vernooij)
FEATURES
* ConfigDict now behaves more like a dictionary.
(Adam 'Cezar' Jenkins, issue #58)
* HTTPGitApplication now takes an optional
`fallback_app` argument. (Jonas Haag, issue #67)
* Support for large pack index files. (Jameson Nash)
TESTING
* Make index entry tests a little bit less strict, to cope with
slightly different behaviour on various platforms.
(Jelmer Vernooij)
* ``setup.py test`` (available when setuptools is installed) now
runs all tests, not just the basic unit tests.
(Jelmer Vernooij)
BUG FIXES
* Commit._deserialize now actually deserializes the current state rather than
the previous one. (Yifan Zhang, issue #59)
* Handle None elements in lists of TreeChange objects. (Alex Holmes)
* Support cloning repositories without HEAD set.
(D-Key, Jelmer Vernooij, issue #69)
* Support ``MemoryRepo.get_config``. (Jelmer Vernooij)
* In ``get_transport_and_path``, pass extra keyword arguments on to
HttpGitClient. (Jelmer Vernooij)
0.8.5 2012-03-29
BUG FIXES
* Avoid use of 'with' in dulwich.index. (Jelmer Vernooij)
* Be a little bit strict about OS behaviour in index tests.
Should fix the tests on Debian GNU/kFreeBSD. (Jelmer Vernooij)
0.8.4 2012-03-28
BUG FIXES
* Options on the same line as sections in config files are now supported.
(Jelmer Vernooij, #920553)
* Only negotiate capabilities that are also supported by the server.
(Rod Cloutier, Risto Kankkunen)
* Fix parsing of invalid timezone offsets with two minus signs.
(Jason R. Coombs, #697828)
* Reset environment variables during tests, to avoid
test isolation leaks reading ~/.gitconfig. (Risto Kankkunen)
TESTS
* $HOME is now explicitly specified for tests that use it to read
``~/.gitconfig``, to prevent test isolation issues.
(Jelmer Vernooij, #920330)
FEATURES
* Additional arguments to get_transport_and_path are now passed
on to the constructor of the transport. (Sam Vilain)
* The WSGI server now transparently handles when a git client submits data
using Content-Encoding: gzip.
(David Blewett, Jelmer Vernooij)
* Add dulwich.index.build_index_from_tree(). (milki)
0.8.3 2012-01-21
FEATURES
* The config parser now supports the git-config file format as
described in git-config(1) and can write git config files.
(Jelmer Vernooij, #531092, #768687)
* ``Repo.do_commit`` will now use the user identity from
.git/config or ~/.gitconfig if none was explicitly specified.
(Jelmer Vernooij)
BUG FIXES
* Allow ``determine_wants`` methods to include the zero sha in their
return value. (Jelmer Vernooij)
0.8.2 2011-12-18
BUG FIXES
* Cope with different zlib buffer sizes in sha1 file parser.
(Jelmer Vernooij)
* Fix get_transport_and_path for HTTP/HTTPS URLs.
(Bruno Renié)
* Avoid calling free_objects() on NULL in error cases. (Chris Eberle)
* Fix use --bare argument to 'dulwich init'. (Chris Eberle)
* Properly abort connections when the determine_wants function
raises an exception. (Jelmer Vernooij, #856769)
* Tweak xcodebuild hack to deal with more error output.
(Jelmer Vernooij, #903840)
FEATURES
* Add support for retrieving tarballs from remote servers.
(Jelmer Vernooij, #379087)
* New method ``update_server_info`` which generates data
for dumb server access. (Jelmer Vernooij, #731235)
0.8.1 2011-10-31
FEATURES
* Repo.do_commit has a new argument 'ref'.
* Repo.do_commit has a new argument 'merge_heads'. (Jelmer Vernooij)
* New ``Repo.get_walker`` method. (Jelmer Vernooij)
* New ``Repo.clone`` method. (Jelmer Vernooij, #725369)
* ``GitClient.send_pack`` now supports the 'side-band-64k' capability.
(Jelmer Vernooij)
* ``HttpGitClient`` which supports the smart server protocol over
HTTP. "dumb" access is not yet supported. (Jelmer Vernooij, #373688)
* Add basic support for alternates. (Jelmer Vernooij, #810429)
CHANGES
* unittest2 or python >= 2.7 is now required for the testsuite.
testtools is no longer supported. (Jelmer Vernooij, #830713)
BUG FIXES
* Fix compilation with older versions of MSVC. (Martin gz)
* Special case 'refs/stash' as a valid ref. (Jelmer Vernooij, #695577)
* Smart protocol clients can now change refs even if they are
not uploading new data. (Jelmer Vernooij, #855993)
* Don't compile C extensions when running in pypy.
(Ronny Pfannschmidt, #881546)
* Use different name for strnlen replacement function to avoid clashing
with system strnlen. (Jelmer Vernooij, #880362)
API CHANGES
* ``Repo.revision_history`` is now deprecated in favor of ``Repo.get_walker``.
(Jelmer Vernooij)
0.8.0 2011-08-07
FEATURES
* New DeltaChainIterator abstract class for quickly iterating all objects in
a pack, with implementations for pack indexing and inflation.
(Dave Borowitz)
* New walk module with a Walker class for customizable commit walking.
(Dave Borowitz)
* New tree_changes_for_merge function in diff_tree. (Dave Borowitz)
* Easy rename detection in RenameDetector even without find_copies_harder.
(Dave Borowitz)
BUG FIXES
* Avoid storing all objects in memory when writing pack.
(Jelmer Vernooij, #813268)
* Support IPv6 for git:// connections. (Jelmer Vernooij, #801543)
* Improve performance of Repo.revision_history(). (Timo Schmid, #535118)
* Fix use of SubprocessWrapper on Windows. (Paulo Madeira, #670035)
* Fix compilation on newer versions of Mac OS X (Lion and up). (Ryan McKern, #794543)
* Prevent raising ValueError for correct refs in RefContainer.__delitem__.
* Correctly return a tuple from MemoryObjectStore.get_raw. (Dave Borowitz)
* Fix a bug in reading the pack checksum when there are fewer than 20 bytes
left in the buffer. (Dave Borowitz)
* Support ~ in git:// URL paths. (Jelmer Vernooij, #813555)
* Make ShaFile.__eq__ work when other is not a ShaFile. (Dave Borowitz)
* ObjectStore.get_graph_walker() now no longer yields the same
revision more than once. This has a significant improvement for
performance when wide revision graphs are involved.
(Jelmer Vernooij, #818168)
* Teach ReceivePackHandler how to read empty packs. (Dave Borowitz)
* Don't send a pack with duplicates of the same object. (Dave Borowitz)
* Teach the server how to serve a clone of an empty repo. (Dave Borowitz)
* Correctly advertise capabilities during receive-pack. (Dave Borowitz)
* Fix add/add and add/rename conflicts in tree_changes_for_merge.
(Dave Borowitz)
* Use correct MIME types in web server. (Dave Borowitz)
API CHANGES
* write_pack no longer takes the num_objects argument and requires an object
to be passed in that is iterable (rather than an iterator) and that
provides __len__. (Jelmer Vernooij)
* write_pack_data has been renamed to write_pack_objects and no longer takes a
num_objects argument. (Jelmer Vernooij)
* take_msb_bytes, read_zlib_chunks, unpack_objects, and
PackStreamReader.read_objects now take an additional argument indicating a
crc32 to compute. (Dave Borowitz)
* PackObjectIterator was removed; its functionality is still exposed by
PackData.iterobjects. (Dave Borowitz)
* Add a sha arg to write_pack_object to incrementally compute a SHA.
(Dave Borowitz)
* Include offset in PackStreamReader results. (Dave Borowitz)
* Move PackStreamReader from server to pack. (Dave Borowitz)
* Extract a check_length_and_checksum, compute_file_sha, and
pack_object_header pack helper functions. (Dave Borowitz)
* Extract a compute_file_sha function. (Dave Borowitz)
* Remove move_in_thin_pack as a separate method; add_thin_pack now completes
the thin pack and moves it in in one step. Remove ThinPackData as well.
(Dave Borowitz)
* Custom buffer size in read_zlib_chunks. (Dave Borowitz)
* New UnpackedObject data class that replaces ad-hoc tuples in the return
value of unpack_object and various DeltaChainIterator methods.
(Dave Borowitz)
* Add a lookup_path convenience method to Tree. (Dave Borowitz)
* Optionally create RenameDetectors without passing in tree SHAs.
(Dave Borowitz)
* Optionally include unchanged entries in RenameDetectors. (Dave Borowitz)
* Optionally pass a RenameDetector to tree_changes. (Dave Borowitz)
* Optionally pass a request object through to server handlers. (Dave Borowitz)
TEST CHANGES
* If setuptools is installed, "python setup.py test" will now run the testsuite.
(Jelmer Vernooij)
* Add a new build_pack test utility for building packs from a simple spec.
(Dave Borowitz)
* Add a new build_commit_graph test utility for building commits from a
simple spec. (Dave Borowitz)
0.7.1 2011-04-12
BUG FIXES
* Fix double decref in _diff_tree.c. (Ted Horst, #715528)
* Fix the build on Windows. (Pascal Quantin)
* Fix get_transport_and_path compatibility with pre-2.6.5 versions of Python.
(Max Bowsher, #707438)
* BaseObjectStore.determine_wants_all no longer breaks on zero SHAs.
(Jelmer Vernooij)
* write_tree_diff() now supports submodules.
(Jelmer Vernooij)
* Fix compilation for XCode 4 and older versions of distutils.sysconfig.
(Daniele Sluijters)
IMPROVEMENTS
* Sphinxified documentation. (Lukasz Balcerzak)
* Add Pack.keep.(Marc Brinkmann)
API CHANGES
* The order of the parameters to Tree.add(name, mode, sha) has changed, and
is now consistent with the rest of Dulwich. Existing code will still
work but print a DeprecationWarning. (Jelmer Vernooij, #663550)
* Tree.entries() is now deprecated in favour of Tree.items() and
Tree.iteritems(). (Jelmer Vernooij)
0.7.0 2011-01-21
FEATURES
* New `dulwich.diff_tree` module for simple content-based rename detection.
(Dave Borowitz)
* Add Tree.items(). (Jelmer Vernooij)
* Add eof() and unread_pkt_line() methods to Protocol. (Dave Borowitz)
* Add write_tree_diff(). (Jelmer Vernooij)
* Add `serve_command` function for git server commands as executables.
(Jelmer Vernooij)
* dulwich.client.get_transport_and_path now supports rsync-style repository URLs.
(Dave Borowitz, #568493)
BUG FIXES
* Correct short-circuiting operation for no-op fetches in the server.
(Dave Borowitz)
* Support parsing git mbox patches without a version tail, as generated by
Mercurial. (Jelmer Vernooij)
* Fix dul-receive-pack and dul-upload-pack. (Jelmer Vernooij)
* Zero-padded file modes in Tree objects no longer trigger an exception but
the check code warns about them. (Augie Fackler, #581064)
* Repo.init() now honors the mkdir flag. (#671159)
* The ref format is now checked when setting a ref rather than when reading it back.
(Dave Borowitz, #653527)
* Make sure pack files are closed correctly. (Tay Ray Chuan)
DOCUMENTATION
* Run the tutorial inside the test suite. (Jelmer Vernooij)
* Reorganized and updated the tutorial. (Jelmer Vernooij, Dave Borowitz, #610550,
#610540)
0.6.2 2010-10-16
BUG FIXES
* HTTP server correctly handles empty CONTENT_LENGTH. (Dave Borowitz)
* Don't error when creating GitFiles with the default mode. (Dave Borowitz)
* ThinPackData.from_file now works with resolve_ext_ref callback.
(Dave Borowitz)
* Provide strnlen() on mingw32 which doesn't have it. (Hans Kolek)
* Set bare=true in the configuratin for bare repositories. (Dirk Neumann)
FEATURES
* Use slots for core objects to save up on memory. (Jelmer Vernooij)
* Web server supports streaming progress/pack output. (Dave Borowitz)
* New public function dulwich.pack.write_pack_header. (Dave Borowitz)
* Distinguish between missing files and read errors in HTTP server.
(Dave Borowitz)
* Initial work on support for fastimport using python-fastimport.
(Jelmer Vernooij)
* New dulwich.pack.MemoryPackIndex class. (Jelmer Vernooij)
* Delegate SHA peeling to the object store. (Dave Borowitz)
TESTS
* Use GitFile when modifying packed-refs in tests. (Dave Borowitz)
* New tests in test_web with better coverage and fewer ad-hoc mocks.
(Dave Borowitz)
* Standardize quote delimiters in test_protocol. (Dave Borowitz)
* Fix use when testtools is installed. (Jelmer Vernooij)
* Add trivial test for write_pack_header. (Jelmer Vernooij)
* Refactor some of dulwich.tests.compat.server_utils. (Dave Borowitz)
* Allow overwriting id property of objects in test utils. (Dave Borowitz)
* Use real in-memory objects rather than stubs for server tests.
(Dave Borowitz)
* Clean up MissingObjectFinder. (Dave Borowitz)
API CHANGES
* ObjectStore.iter_tree_contents now walks contents in depth-first, sorted
order. (Dave Borowitz)
* ObjectStore.iter_tree_contents can optionally yield tree objects as well.
(Dave Borowitz).
* Add side-band-64k support to ReceivePackHandler. (Dave Borowitz)
* Change server capabilities methods to classmethods. (Dave Borowitz)
* Tweak server handler injection. (Dave Borowitz)
* PackIndex1 and PackIndex2 now subclass FilePackIndex, which is
itself a subclass of PackIndex. (Jelmer Vernooij)
DOCUMENTATION
* Add docstrings for various functions in dulwich.objects. (Jelmer Vernooij)
* Clean up docstrings in dulwich.protocol. (Dave Borowitz)
* Explicitly specify allowed protocol commands to
ProtocolGraphWalker.read_proto_line. (Dave Borowitz)
* Add utility functions to DictRefsContainer. (Dave Borowitz)
0.6.1 2010-07-22
BUG FIXES
* Fix memory leak in C implementation of sorted_tree_items. (Dave Borowitz)
* Use correct path separators for named repo files. (Dave Borowitz)
* python > 2.7 and testtools-based test runners will now also pick up skipped
tests correctly. (Jelmer Vernooij)
FEATURES
* Move named file initilization to BaseRepo. (Dave Borowitz)
* Add logging utilities and git/HTTP server logging. (Dave Borowitz)
* The GitClient interface has been cleaned up and instances are now reusable.
(Augie Fackler)
* Allow overriding paths to executables in GitSSHClient.
(Ross Light, Jelmer Vernooij, #585204)
* Add PackBasedObjectStore.pack_loose_objects(). (Jelmer Vernooij)
TESTS
* Add tests for sorted_tree_items and C implementation. (Dave Borowitz)
* Add a MemoryRepo that stores everything in memory. (Dave Borowitz)
* Quiet logging output from web tests. (Dave Borowitz)
* More flexible version checking for compat tests. (Dave Borowitz)
* Compat tests for servers with and without side-band-64k. (Dave Borowitz)
CLEANUP
* Clean up file headers. (Dave Borowitz)
TESTS
* Use GitFile when modifying packed-refs in tests. (Dave Borowitz)
API CHANGES
* dulwich.pack.write_pack_index_v{1,2} now take a file-like object
rather than a filename. (Jelmer Vernooij)
* Make dul-daemon/dul-web trivial wrappers around server functionality.
(Dave Borowitz)
* Move reference WSGI handler to web.py. (Dave Borowitz)
* Factor out _report_status in ReceivePackHandler. (Dave Borowitz)
* Factor out a function to convert a line to a pkt-line. (Dave Borowitz)
0.6.0 2010-05-22
note: This list is most likely incomplete for 0.6.0.
BUG FIXES
* Fix ReceivePackHandler to disallow removing refs without delete-refs.
(Dave Borowitz)
* Deal with capabilities required by the client, even if they
can not be disabled in the server. (Dave Borowitz)
* Fix trailing newlines in generated patch files.
(Jelmer Vernooij)
* Implement RefsContainer.__contains__. (Jelmer Vernooij)
* Cope with \r in ref files on Windows. (
http://github.com/jelmer/dulwich/issues/#issue/13, Jelmer Vernooij)
* Fix GitFile breakage on Windows. (Anatoly Techtonik, #557585)
* Support packed ref deletion with no peeled refs. (Augie Fackler)
* Fix send pack when there is nothing to fetch. (Augie Fackler)
* Fix fetch if no progress function is specified. (Augie Fackler)
* Allow double-staging of files that are deleted in the index.
(Dave Borowitz)
* Fix RefsContainer.add_if_new to support dangling symrefs.
(Dave Borowitz)
* Non-existant index files in non-bare repositories are now treated as
empty. (Dave Borowitz)
* Always update ShaFile.id when the contents of the object get changed.
(Jelmer Vernooij)
* Various Python2.4-compatibility fixes. (Dave Borowitz)
* Fix thin pack handling. (Dave Borowitz)
FEATURES
* Add include-tag capability to server. (Dave Borowitz)
* New dulwich.fastexport module that can generate fastexport
streams. (Jelmer Vernooij)
* Implemented BaseRepo.__contains__. (Jelmer Vernooij)
* Add __setitem__ to DictRefsContainer. (Dave Borowitz)
* Overall improvements checking Git objects. (Dave Borowitz)
* Packs are now verified while they are received. (Dave Borowitz)
TESTS
* Add framework for testing compatibility with C Git. (Dave Borowitz)
* Add various tests for the use of non-bare repositories. (Dave Borowitz)
* Cope with diffstat not being available on all platforms.
(Tay Ray Chuan, Jelmer Vernooij)
* Add make_object and make_commit convenience functions to test utils.
(Dave Borowitz)
API BREAKAGES
* The 'committer' and 'message' arguments to Repo.do_commit() have
been swapped. 'committer' is now optional. (Jelmer Vernooij)
* Repo.get_blob, Repo.commit, Repo.tag and Repo.tree are now deprecated.
(Jelmer Vernooij)
* RefsContainer.set_ref() was renamed to RefsContainer.set_symbolic_ref(),
for clarity. (Jelmer Vernooij)
API CHANGES
* The primary serialization APIs in dulwich.objects now work
with chunks of strings rather than with full-text strings.
(Jelmer Vernooij)
0.5.02010-03-03
BUG FIXES
* Support custom fields in commits (readonly). (Jelmer Vernooij)
* Improved ref handling. (Dave Borowitz)
* Rework server protocol to be smarter and interoperate with cgit client.
(Dave Borowitz)
* Add a GitFile class that uses the same locking protocol for writes as
cgit. (Dave Borowitz)
* Cope with forward slashes correctly in the index on Windows.
(Jelmer Vernooij, #526793)
FEATURES
* --pure option to setup.py to allow building/installing without the C
extensions. (Hal Wine, Anatoly Techtonik, Jelmer Vernooij, #434326)
* Implement Repo.get_config(). (Jelmer Vernooij, Augie Fackler)
* HTTP dumb and smart server. (Dave Borowitz)
* Add abstract baseclass for Repo that does not require file system
operations. (Dave Borowitz)
0.4.1 2010-01-03
FEATURES
* Add ObjectStore.iter_tree_contents(). (Jelmer Vernooij)
* Add Index.changes_from_tree(). (Jelmer Vernooij)
* Add ObjectStore.tree_changes(). (Jelmer Vernooij)
* Add functionality for writing patches in dulwich.patch.
(Jelmer Vernooij)
0.4.0 2009-10-07
DOCUMENTATION
* Added tutorial.
API CHANGES
* dulwich.object_store.tree_lookup_path will now return the mode and
sha of the object found rather than the object itself.
BUG FIXES
* Use binascii.hexlify / binascii.unhexlify for better performance.
* Cope with extra unknown data in index files by ignoring it (for now).
* Add proper error message when server unexpectedly hangs up. (#415843)
* Correctly write opcode for equal in create_delta.
0.3.3 2009-07-23
FEATURES
* Implement ShaFile.__hash__().
* Implement Tree.__len__()
BUG FIXES
* Check for 'objects' and 'refs' directories
when looking for a Git repository. (#380818)
0.3.2 2009-05-20
BUG FIXES
* Support the encoding field in Commits.
* Some Windows compatibility fixes.
* Fixed several issues in commit support.
FEATURES
* Basic support for handling submodules.
0.3.1 2009-05-13
FEATURES
* Implemented Repo.__getitem__, Repo.__setitem__ and Repo.__delitem__ to
access content.
API CHANGES
* Removed Repo.set_ref, Repo.remove_ref, Repo.tags, Repo.get_refs and
Repo.heads in favor of Repo.refs, a dictionary-like object for accessing
refs.
BUG FIXES
* Removed import of 'sha' module in objects.py, which was causing
deprecation warnings on Python 2.6.
0.3.0 2009-05-10
FEATURES
* A new function 'commit_tree' has been added that can commit a tree
based on an index.
BUG FIXES
* The memory usage when generating indexes has been significantly reduced.
* A memory leak in the C implementation of parse_tree has been fixed.
* The send-pack smart server command now works. (Thanks Scott Chacon)
* The handling of short timestamps (less than 10 digits) has been fixed.
* The handling of timezones has been fixed.
0.2.1 2009-04-30
BUG FIXES
* Fix compatibility with Python2.4.
0.2.0 2009-04-30
FEATURES
* Support for activity reporting in smart protocol client.
* Optional C extensions for better performance in a couple of
places that are performance-critical.
0.1.1 2009-03-13
BUG FIXES
* Fixed regression in Repo.find_missing_objects()
* Don't fetch ^{} objects from remote hosts, as requesting them
causes a hangup.
* Always write pack to disk completely before calculating checksum.
FEATURES
* Allow disabling thin packs when talking to remote hosts.
0.1.0 2009-01-24
* Initial release.
diff --git a/dulwich/tests/test_walk.py b/dulwich/tests/test_walk.py
index 5627ee90..de9098d9 100644
--- a/dulwich/tests/test_walk.py
+++ b/dulwich/tests/test_walk.py
@@ -1,543 +1,550 @@
# test_walk.py -- Tests for commit walking functionality.
# Copyright (C) 2010 Google, Inc.
#
# Dulwich is dual-licensed under the Apache License, Version 2.0 and the GNU
# General Public License as public by the Free Software Foundation; version 2.0
# or (at your option) any later version. You can redistribute it and/or
# modify it under the terms of either of these two licenses.
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# You should have received a copy of the licenses; if not, see
# for a copy of the GNU General Public License
# and for a copy of the Apache
# License, Version 2.0.
#
"""Tests for commit walking functionality."""
from itertools import (
permutations,
)
from dulwich.diff_tree import (
CHANGE_ADD,
CHANGE_MODIFY,
CHANGE_RENAME,
TreeChange,
RenameDetector,
)
from dulwich.errors import (
MissingCommitError,
)
from dulwich.object_store import (
MemoryObjectStore,
)
from dulwich.objects import (
Commit,
Blob,
)
from dulwich.walk import (
ORDER_TOPO,
WalkEntry,
Walker,
_topo_reorder
)
from dulwich.tests import TestCase
from dulwich.tests.utils import (
F,
make_object,
+ make_tag,
build_commit_graph,
)
class TestWalkEntry(object):
def __init__(self, commit, changes):
self.commit = commit
self.changes = changes
def __repr__(self):
return '' % (
self.commit.id, self.changes)
def __eq__(self, other):
if not isinstance(other, WalkEntry) or self.commit != other.commit:
return False
if self.changes is None:
return True
return self.changes == other.changes()
class WalkerTest(TestCase):
def setUp(self):
super(WalkerTest, self).setUp()
self.store = MemoryObjectStore()
def make_commits(self, commit_spec, **kwargs):
times = kwargs.pop('times', [])
attrs = kwargs.pop('attrs', {})
for i, t in enumerate(times):
attrs.setdefault(i + 1, {})['commit_time'] = t
return build_commit_graph(self.store, commit_spec, attrs=attrs,
**kwargs)
def make_linear_commits(self, num_commits, **kwargs):
commit_spec = []
for i in range(1, num_commits + 1):
c = [i]
if i > 1:
c.append(i - 1)
commit_spec.append(c)
return self.make_commits(commit_spec, **kwargs)
def assertWalkYields(self, expected, *args, **kwargs):
walker = Walker(self.store, *args, **kwargs)
expected = list(expected)
for i, entry in enumerate(expected):
if isinstance(entry, Commit):
expected[i] = TestWalkEntry(entry, None)
actual = list(walker)
self.assertEqual(expected, actual)
+ def test_tag(self):
+ c1, c2, c3 = self.make_linear_commits(3)
+ t2 = make_tag(target=c2)
+ self.store.add_object(t2)
+ self.assertWalkYields([c2, c1], [t2.id])
+
def test_linear(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([c1], [c1.id])
self.assertWalkYields([c2, c1], [c2.id])
self.assertWalkYields([c3, c2, c1], [c3.id])
self.assertWalkYields([c3, c2, c1], [c3.id, c1.id])
self.assertWalkYields([c3, c2], [c3.id], exclude=[c1.id])
self.assertWalkYields([c3, c2], [c3.id, c1.id], exclude=[c1.id])
self.assertWalkYields([c3], [c3.id, c1.id], exclude=[c2.id])
def test_missing(self):
cs = list(reversed(self.make_linear_commits(20)))
self.assertWalkYields(cs, [cs[0].id])
# Exactly how close we can get to a missing commit depends on our
# implementation (in particular the choice of _MAX_EXTRA_COMMITS), but
# we should at least be able to walk some history in a broken repo.
del self.store[cs[-1].id]
for i in range(1, 11):
self.assertWalkYields(cs[:i], [cs[0].id], max_entries=i)
self.assertRaises(MissingCommitError, Walker, self.store, [cs[-1].id])
def test_branch(self):
c1, x2, x3, y4 = self.make_commits([[1], [2, 1], [3, 2], [4, 1]])
self.assertWalkYields([x3, x2, c1], [x3.id])
self.assertWalkYields([y4, c1], [y4.id])
self.assertWalkYields([y4, x2, c1], [y4.id, x2.id])
self.assertWalkYields([y4, x2], [y4.id, x2.id], exclude=[c1.id])
self.assertWalkYields([y4, x3], [y4.id, x3.id], exclude=[x2.id])
self.assertWalkYields([y4], [y4.id], exclude=[x3.id])
self.assertWalkYields([x3, x2], [x3.id], exclude=[y4.id])
def test_merge(self):
c1, c2, c3, c4 = self.make_commits([[1], [2, 1], [3, 1], [4, 2, 3]])
self.assertWalkYields([c4, c3, c2, c1], [c4.id])
self.assertWalkYields([c3, c1], [c3.id])
self.assertWalkYields([c2, c1], [c2.id])
self.assertWalkYields([c4, c3], [c4.id], exclude=[c2.id])
self.assertWalkYields([c4, c2], [c4.id], exclude=[c3.id])
def test_reverse(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([c1, c2, c3], [c3.id], reverse=True)
def test_max_entries(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([c3, c2, c1], [c3.id], max_entries=3)
self.assertWalkYields([c3, c2], [c3.id], max_entries=2)
self.assertWalkYields([c3], [c3.id], max_entries=1)
def test_reverse_after_max_entries(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([c1, c2, c3], [c3.id], max_entries=3,
reverse=True)
self.assertWalkYields([c2, c3], [c3.id], max_entries=2, reverse=True)
self.assertWalkYields([c3], [c3.id], max_entries=1, reverse=True)
def test_changes_one_parent(self):
blob_a1 = make_object(Blob, data=b'a1')
blob_a2 = make_object(Blob, data=b'a2')
blob_b2 = make_object(Blob, data=b'b2')
c1, c2 = self.make_linear_commits(
2, trees={1: [(b'a', blob_a1)],
2: [(b'a', blob_a2), (b'b', blob_b2)]})
e1 = TestWalkEntry(c1, [TreeChange.add((b'a', F, blob_a1.id))])
e2 = TestWalkEntry(c2, [TreeChange(CHANGE_MODIFY, (b'a', F, blob_a1.id),
(b'a', F, blob_a2.id)),
TreeChange.add((b'b', F, blob_b2.id))])
self.assertWalkYields([e2, e1], [c2.id])
def test_changes_multiple_parents(self):
blob_a1 = make_object(Blob, data=b'a1')
blob_b2 = make_object(Blob, data=b'b2')
blob_a3 = make_object(Blob, data=b'a3')
c1, c2, c3 = self.make_commits(
[[1], [2], [3, 1, 2]],
trees={1: [(b'a', blob_a1)], 2: [(b'b', blob_b2)],
3: [(b'a', blob_a3), (b'b', blob_b2)]})
# a is a modify/add conflict and b is not conflicted.
changes = [[
TreeChange(CHANGE_MODIFY, (b'a', F, blob_a1.id), (b'a', F, blob_a3.id)),
TreeChange.add((b'a', F, blob_a3.id)),
]]
self.assertWalkYields([TestWalkEntry(c3, changes)], [c3.id],
exclude=[c1.id, c2.id])
def test_path_matches(self):
walker = Walker(None, [], paths=[b'foo', b'bar', b'baz/quux'])
self.assertTrue(walker._path_matches(b'foo'))
self.assertTrue(walker._path_matches(b'foo/a'))
self.assertTrue(walker._path_matches(b'foo/a/b'))
self.assertTrue(walker._path_matches(b'bar'))
self.assertTrue(walker._path_matches(b'baz/quux'))
self.assertTrue(walker._path_matches(b'baz/quux/a'))
self.assertFalse(walker._path_matches(None))
self.assertFalse(walker._path_matches(b'oops'))
self.assertFalse(walker._path_matches(b'fool'))
self.assertFalse(walker._path_matches(b'baz'))
self.assertFalse(walker._path_matches(b'baz/quu'))
def test_paths(self):
blob_a1 = make_object(Blob, data=b'a1')
blob_b2 = make_object(Blob, data=b'b2')
blob_a3 = make_object(Blob, data=b'a3')
blob_b3 = make_object(Blob, data=b'b3')
c1, c2, c3 = self.make_linear_commits(
3, trees={1: [(b'a', blob_a1)],
2: [(b'a', blob_a1), (b'x/b', blob_b2)],
3: [(b'a', blob_a3), (b'x/b', blob_b3)]})
self.assertWalkYields([c3, c2, c1], [c3.id])
self.assertWalkYields([c3, c1], [c3.id], paths=[b'a'])
self.assertWalkYields([c3, c2], [c3.id], paths=[b'x/b'])
# All changes are included, not just for requested paths.
changes = [
TreeChange(CHANGE_MODIFY, (b'a', F, blob_a1.id),
(b'a', F, blob_a3.id)),
TreeChange(CHANGE_MODIFY, (b'x/b', F, blob_b2.id),
(b'x/b', F, blob_b3.id)),
]
self.assertWalkYields([TestWalkEntry(c3, changes)], [c3.id],
max_entries=1, paths=[b'a'])
def test_paths_subtree(self):
blob_a = make_object(Blob, data=b'a')
blob_b = make_object(Blob, data=b'b')
c1, c2, c3 = self.make_linear_commits(
3, trees={1: [(b'x/a', blob_a)],
2: [(b'b', blob_b), (b'x/a', blob_a)],
3: [(b'b', blob_b), (b'x/a', blob_a), (b'x/b', blob_b)]})
self.assertWalkYields([c2], [c3.id], paths=[b'b'])
self.assertWalkYields([c3, c1], [c3.id], paths=[b'x'])
def test_paths_max_entries(self):
blob_a = make_object(Blob, data=b'a')
blob_b = make_object(Blob, data=b'b')
c1, c2 = self.make_linear_commits(
2, trees={1: [(b'a', blob_a)],
2: [(b'a', blob_a), (b'b', blob_b)]})
self.assertWalkYields([c2], [c2.id], paths=[b'b'], max_entries=1)
self.assertWalkYields([c1], [c1.id], paths=[b'a'], max_entries=1)
def test_paths_merge(self):
blob_a1 = make_object(Blob, data=b'a1')
blob_a2 = make_object(Blob, data=b'a2')
blob_a3 = make_object(Blob, data=b'a3')
x1, y2, m3, m4 = self.make_commits(
[[1], [2], [3, 1, 2], [4, 1, 2]],
trees={1: [(b'a', blob_a1)],
2: [(b'a', blob_a2)],
3: [(b'a', blob_a3)],
4: [(b'a', blob_a1)]}) # Non-conflicting
self.assertWalkYields([m3, y2, x1], [m3.id], paths=[b'a'])
self.assertWalkYields([y2, x1], [m4.id], paths=[b'a'])
def test_changes_with_renames(self):
blob = make_object(Blob, data=b'blob')
c1, c2 = self.make_linear_commits(
2, trees={1: [(b'a', blob)], 2: [(b'b', blob)]})
entry_a = (b'a', F, blob.id)
entry_b = (b'b', F, blob.id)
changes_without_renames = [TreeChange.delete(entry_a),
TreeChange.add(entry_b)]
changes_with_renames = [TreeChange(CHANGE_RENAME, entry_a, entry_b)]
self.assertWalkYields(
[TestWalkEntry(c2, changes_without_renames)], [c2.id], max_entries=1)
detector = RenameDetector(self.store)
self.assertWalkYields(
[TestWalkEntry(c2, changes_with_renames)], [c2.id], max_entries=1,
rename_detector=detector)
def test_follow_rename(self):
blob = make_object(Blob, data=b'blob')
names = [b'a', b'a', b'b', b'b', b'c', b'c']
trees = dict((i + 1, [(n, blob, F)]) for i, n in enumerate(names))
c1, c2, c3, c4, c5, c6 = self.make_linear_commits(6, trees=trees)
self.assertWalkYields([c5], [c6.id], paths=[b'c'])
e = lambda n: (n, F, blob.id)
self.assertWalkYields(
[TestWalkEntry(c5, [TreeChange(CHANGE_RENAME, e(b'b'), e(b'c'))]),
TestWalkEntry(c3, [TreeChange(CHANGE_RENAME, e(b'a'), e(b'b'))]),
TestWalkEntry(c1, [TreeChange.add(e(b'a'))])],
[c6.id], paths=[b'c'], follow=True)
def test_follow_rename_remove_path(self):
blob = make_object(Blob, data=b'blob')
_, _, _, c4, c5, c6 = self.make_linear_commits(
6, trees={1: [(b'a', blob), (b'c', blob)],
2: [],
3: [],
4: [(b'b', blob)],
5: [(b'a', blob)],
6: [(b'c', blob)]})
e = lambda n: (n, F, blob.id)
# Once the path changes to b, we aren't interested in a or c anymore.
self.assertWalkYields(
[TestWalkEntry(c6, [TreeChange(CHANGE_RENAME, e(b'a'), e(b'c'))]),
TestWalkEntry(c5, [TreeChange(CHANGE_RENAME, e(b'b'), e(b'a'))]),
TestWalkEntry(c4, [TreeChange.add(e(b'b'))])],
[c6.id], paths=[b'c'], follow=True)
def test_since(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([c3, c2, c1], [c3.id], since=-1)
self.assertWalkYields([c3, c2, c1], [c3.id], since=0)
self.assertWalkYields([c3, c2], [c3.id], since=1)
self.assertWalkYields([c3, c2], [c3.id], since=99)
self.assertWalkYields([c3, c2], [c3.id], since=100)
self.assertWalkYields([c3], [c3.id], since=101)
self.assertWalkYields([c3], [c3.id], since=199)
self.assertWalkYields([c3], [c3.id], since=200)
self.assertWalkYields([], [c3.id], since=201)
self.assertWalkYields([], [c3.id], since=300)
def test_until(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([], [c3.id], until=-1)
self.assertWalkYields([c1], [c3.id], until=0)
self.assertWalkYields([c1], [c3.id], until=1)
self.assertWalkYields([c1], [c3.id], until=99)
self.assertWalkYields([c2, c1], [c3.id], until=100)
self.assertWalkYields([c2, c1], [c3.id], until=101)
self.assertWalkYields([c2, c1], [c3.id], until=199)
self.assertWalkYields([c3, c2, c1], [c3.id], until=200)
self.assertWalkYields([c3, c2, c1], [c3.id], until=201)
self.assertWalkYields([c3, c2, c1], [c3.id], until=300)
def test_since_until(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([], [c3.id], since=100, until=99)
self.assertWalkYields([c3, c2, c1], [c3.id], since=-1, until=201)
self.assertWalkYields([c2], [c3.id], since=100, until=100)
self.assertWalkYields([c2], [c3.id], since=50, until=150)
def test_since_over_scan(self):
commits = self.make_linear_commits(
11, times=[9, 0, 1, 2, 3, 4, 5, 8, 6, 7, 9])
c8, _, c10, c11 = commits[-4:]
del self.store[commits[0].id]
# c9 is older than we want to walk, but is out of order with its parent,
# so we need to walk past it to get to c8.
# c1 would also match, but we've deleted it, and it should get pruned
# even with over-scanning.
self.assertWalkYields([c11, c10, c8], [c11.id], since=7)
def assertTopoOrderEqual(self, expected_commits, commits):
entries = [TestWalkEntry(c, None) for c in commits]
actual_ids = [e.commit.id for e in list(_topo_reorder(entries))]
self.assertEqual([c.id for c in expected_commits], actual_ids)
def test_topo_reorder_linear(self):
commits = self.make_linear_commits(5)
commits.reverse()
for perm in permutations(commits):
self.assertTopoOrderEqual(commits, perm)
def test_topo_reorder_multiple_parents(self):
c1, c2, c3 = self.make_commits([[1], [2], [3, 1, 2]])
# Already sorted, so totally FIFO.
self.assertTopoOrderEqual([c3, c2, c1], [c3, c2, c1])
self.assertTopoOrderEqual([c3, c1, c2], [c3, c1, c2])
# c3 causes one parent to be yielded.
self.assertTopoOrderEqual([c3, c2, c1], [c2, c3, c1])
self.assertTopoOrderEqual([c3, c1, c2], [c1, c3, c2])
# c3 causes both parents to be yielded.
self.assertTopoOrderEqual([c3, c2, c1], [c1, c2, c3])
self.assertTopoOrderEqual([c3, c2, c1], [c2, c1, c3])
def test_topo_reorder_multiple_children(self):
c1, c2, c3 = self.make_commits([[1], [2, 1], [3, 1]])
# c2 and c3 are FIFO but c1 moves to the end.
self.assertTopoOrderEqual([c3, c2, c1], [c3, c2, c1])
self.assertTopoOrderEqual([c3, c2, c1], [c3, c1, c2])
self.assertTopoOrderEqual([c3, c2, c1], [c1, c3, c2])
self.assertTopoOrderEqual([c2, c3, c1], [c2, c3, c1])
self.assertTopoOrderEqual([c2, c3, c1], [c2, c1, c3])
self.assertTopoOrderEqual([c2, c3, c1], [c1, c2, c3])
def test_out_of_order_children(self):
c1, c2, c3, c4, c5 = self.make_commits(
[[1], [2, 1], [3, 2], [4, 1], [5, 3, 4]],
times=[2, 1, 3, 4, 5])
self.assertWalkYields([c5, c4, c3, c1, c2], [c5.id])
self.assertWalkYields([c5, c4, c3, c2, c1], [c5.id], order=ORDER_TOPO)
def test_out_of_order_with_exclude(self):
# Create the following graph:
# c1-------x2---m6
# \ /
# \-y3--y4-/--y5
# Due to skew, y5 is the oldest commit.
c1, x2, y3, y4, y5, m6 = self.make_commits(
[[1], [2, 1], [3, 1], [4, 3], [5, 4], [6, 2, 4]],
times=[2, 3, 4, 5, 1, 6])
self.assertWalkYields([m6, y4, y3, x2, c1], [m6.id])
# Ensure that c1..y4 get excluded even though they're popped from the
# priority queue long before y5.
self.assertWalkYields([m6, x2], [m6.id], exclude=[y5.id])
def test_empty_walk(self):
c1, c2, c3 = self.make_linear_commits(3)
self.assertWalkYields([], [c3.id], exclude=[c3.id])
class WalkEntryTest(TestCase):
def setUp(self):
super(WalkEntryTest, self).setUp()
self.store = MemoryObjectStore()
def make_commits(self, commit_spec, **kwargs):
times = kwargs.pop('times', [])
attrs = kwargs.pop('attrs', {})
for i, t in enumerate(times):
attrs.setdefault(i + 1, {})['commit_time'] = t
return build_commit_graph(self.store, commit_spec, attrs=attrs,
**kwargs)
def make_linear_commits(self, num_commits, **kwargs):
commit_spec = []
for i in range(1, num_commits + 1):
c = [i]
if i > 1:
c.append(i - 1)
commit_spec.append(c)
return self.make_commits(commit_spec, **kwargs)
def test_all_changes(self):
# Construct a commit with 2 files in different subdirectories.
blob_a = make_object(Blob, data=b'a')
blob_b = make_object(Blob, data=b'b')
c1 = self.make_linear_commits(
1,
trees={1: [(b'x/a', blob_a), (b'y/b', blob_b)]},
)[0]
# Get the WalkEntry for the commit.
walker = Walker(self.store, c1.id)
walker_entry = list(walker)[0]
changes = walker_entry.changes()
# Compare the changes with the expected values.
entry_a = (b'x/a', F, blob_a.id)
entry_b = (b'y/b', F, blob_b.id)
self.assertEqual(
[TreeChange.add(entry_a),
TreeChange.add(entry_b)],
changes,
)
def test_all_with_merge(self):
blob_a = make_object(Blob, data=b'a')
blob_a2 = make_object(Blob, data=b'a2')
blob_b = make_object(Blob, data=b'b')
blob_b2 = make_object(Blob, data=b'b2')
x1, y2, m3 = self.make_commits(
[[1], [2], [3, 1, 2]],
trees={1: [(b'x/a', blob_a)],
2: [(b'y/b', blob_b)],
3: [(b'x/a', blob_a2), (b'y/b', blob_b2)]})
# Get the WalkEntry for the merge commit.
walker = Walker(self.store, m3.id)
entries = list(walker)
walker_entry = entries[0]
self.assertEqual(walker_entry.commit.id, m3.id)
changes = walker_entry.changes()
self.assertEqual(2, len(changes))
entry_a = (b'x/a', F, blob_a.id)
entry_a2 = (b'x/a', F, blob_a2.id)
entry_b = (b'y/b', F, blob_b.id)
entry_b2 = (b'y/b', F, blob_b2.id)
self.assertEqual(
[[TreeChange(CHANGE_MODIFY, entry_a, entry_a2),
TreeChange.add(entry_a2)],
[TreeChange.add(entry_b2),
TreeChange(CHANGE_MODIFY, entry_b, entry_b2)]],
changes,
)
def test_filter_changes(self):
# Construct a commit with 2 files in different subdirectories.
blob_a = make_object(Blob, data=b'a')
blob_b = make_object(Blob, data=b'b')
c1 = self.make_linear_commits(
1,
trees={1: [(b'x/a', blob_a), (b'y/b', blob_b)]},
)[0]
# Get the WalkEntry for the commit.
walker = Walker(self.store, c1.id)
walker_entry = list(walker)[0]
changes = walker_entry.changes(path_prefix=b'x')
# Compare the changes with the expected values.
entry_a = (b'a', F, blob_a.id)
self.assertEqual(
[TreeChange.add(entry_a)],
changes,
)
def test_filter_with_merge(self):
blob_a = make_object(Blob, data=b'a')
blob_a2 = make_object(Blob, data=b'a2')
blob_b = make_object(Blob, data=b'b')
blob_b2 = make_object(Blob, data=b'b2')
x1, y2, m3 = self.make_commits(
[[1], [2], [3, 1, 2]],
trees={1: [(b'x/a', blob_a)],
2: [(b'y/b', blob_b)],
3: [(b'x/a', blob_a2), (b'y/b', blob_b2)]})
# Get the WalkEntry for the merge commit.
walker = Walker(self.store, m3.id)
entries = list(walker)
walker_entry = entries[0]
self.assertEqual(walker_entry.commit.id, m3.id)
changes = walker_entry.changes(b'x')
self.assertEqual(1, len(changes))
entry_a = (b'a', F, blob_a.id)
entry_a2 = (b'a', F, blob_a2.id)
self.assertEqual(
[[TreeChange(CHANGE_MODIFY, entry_a, entry_a2)]],
changes,
)
diff --git a/dulwich/walk.py b/dulwich/walk.py
index f5696543..041cca3e 100644
--- a/dulwich/walk.py
+++ b/dulwich/walk.py
@@ -1,405 +1,414 @@
# walk.py -- General implementation of walking commits and their contents.
# Copyright (C) 2010 Google, Inc.
#
# Dulwich is dual-licensed under the Apache License, Version 2.0 and the GNU
# General Public License as public by the Free Software Foundation; version 2.0
# or (at your option) any later version. You can redistribute it and/or
# modify it under the terms of either of these two licenses.
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# You should have received a copy of the licenses; if not, see
# for a copy of the GNU General Public License
# and for a copy of the Apache
# License, Version 2.0.
#
"""General implementation of walking commits and their contents."""
from collections import defaultdict
import collections
import heapq
from itertools import chain
from dulwich.diff_tree import (
RENAME_CHANGE_TYPES,
tree_changes,
tree_changes_for_merge,
RenameDetector,
)
from dulwich.errors import (
MissingCommitError,
)
+from dulwich.objects import (
+ Commit,
+ Tag,
+ )
ORDER_DATE = 'date'
ORDER_TOPO = 'topo'
ALL_ORDERS = (ORDER_DATE, ORDER_TOPO)
# Maximum number of commits to walk past a commit time boundary.
_MAX_EXTRA_COMMITS = 5
class WalkEntry(object):
"""Object encapsulating a single result from a walk."""
def __init__(self, walker, commit):
self.commit = commit
self._store = walker.store
self._get_parents = walker.get_parents
self._changes = {}
self._rename_detector = walker.rename_detector
def changes(self, path_prefix=None):
"""Get the tree changes for this entry.
:param path_prefix: Portion of the path in the repository to
use to filter changes. Must be a directory name. Must be
a full, valid, path reference (no partial names or wildcards).
:return: For commits with up to one parent, a list of TreeChange
objects; if the commit has no parents, these will be relative to the
empty tree. For merge commits, a list of lists of TreeChange
objects; see dulwich.diff.tree_changes_for_merge.
"""
cached = self._changes.get(path_prefix)
if cached is None:
commit = self.commit
if not self._get_parents(commit):
changes_func = tree_changes
parent = None
elif len(self._get_parents(commit)) == 1:
changes_func = tree_changes
parent = self._store[self._get_parents(commit)[0]].tree
if path_prefix:
mode, subtree_sha = parent.lookup_path(
self._store.__getitem__,
path_prefix,
)
parent = self._store[subtree_sha]
else:
changes_func = tree_changes_for_merge
parent = [self._store[p].tree for p in self._get_parents(commit)]
if path_prefix:
parent_trees = [self._store[p] for p in parent]
parent = []
for p in parent_trees:
try:
mode, st = p.lookup_path(
self._store.__getitem__,
path_prefix,
)
except KeyError:
pass
else:
parent.append(st)
commit_tree_sha = commit.tree
if path_prefix:
commit_tree = self._store[commit_tree_sha]
mode, commit_tree_sha = commit_tree.lookup_path(
self._store.__getitem__,
path_prefix,
)
cached = list(changes_func(
self._store, parent, commit_tree_sha,
rename_detector=self._rename_detector))
self._changes[path_prefix] = cached
return self._changes[path_prefix]
def __repr__(self):
return '' % (
self.commit.id, self.changes())
class _CommitTimeQueue(object):
"""Priority queue of WalkEntry objects by commit time."""
def __init__(self, walker):
self._walker = walker
self._store = walker.store
self._get_parents = walker.get_parents
self._excluded = walker.excluded
self._pq = []
self._pq_set = set()
self._seen = set()
self._done = set()
self._min_time = walker.since
self._last = None
self._extra_commits_left = _MAX_EXTRA_COMMITS
self._is_finished = False
for commit_id in chain(walker.include, walker.excluded):
self._push(commit_id)
- def _push(self, commit_id):
+ def _push(self, object_id):
try:
- commit = self._store[commit_id]
+ obj = self._store[object_id]
except KeyError:
- raise MissingCommitError(commit_id)
- if commit_id not in self._pq_set and commit_id not in self._done:
+ raise MissingCommitError(object_id)
+ if isinstance(obj, Tag):
+ self._push(obj.object[1])
+ return
+ # TODO(jelmer): What to do about non-Commit and non-Tag objects?
+ commit = obj
+ if commit.id not in self._pq_set and commit.id not in self._done:
heapq.heappush(self._pq, (-commit.commit_time, commit))
- self._pq_set.add(commit_id)
- self._seen.add(commit_id)
+ self._pq_set.add(commit.id)
+ self._seen.add(commit.id)
def _exclude_parents(self, commit):
excluded = self._excluded
seen = self._seen
todo = [commit]
while todo:
commit = todo.pop()
for parent in self._get_parents(commit):
if parent not in excluded and parent in seen:
# TODO: This is inefficient unless the object store does
# some caching (which DiskObjectStore currently does not).
# We could either add caching in this class or pass around
# parsed queue entry objects instead of commits.
todo.append(self._store[parent])
excluded.add(parent)
def next(self):
if self._is_finished:
return None
while self._pq:
_, commit = heapq.heappop(self._pq)
sha = commit.id
self._pq_set.remove(sha)
if sha in self._done:
continue
self._done.add(sha)
for parent_id in self._get_parents(commit):
self._push(parent_id)
reset_extra_commits = True
is_excluded = sha in self._excluded
if is_excluded:
self._exclude_parents(commit)
if self._pq and all(c.id in self._excluded
for _, c in self._pq):
_, n = self._pq[0]
if self._last and n.commit_time >= self._last.commit_time:
# If the next commit is newer than the last one, we need
# to keep walking in case its parents (which we may not
# have seen yet) are excluded. This gives the excluded
# set a chance to "catch up" while the commit is still
# in the Walker's output queue.
reset_extra_commits = True
else:
reset_extra_commits = False
if (self._min_time is not None and
commit.commit_time < self._min_time):
# We want to stop walking at min_time, but commits at the
# boundary may be out of order with respect to their parents. So
# we walk _MAX_EXTRA_COMMITS more commits once we hit this
# boundary.
reset_extra_commits = False
if reset_extra_commits:
# We're not at a boundary, so reset the counter.
self._extra_commits_left = _MAX_EXTRA_COMMITS
else:
self._extra_commits_left -= 1
if not self._extra_commits_left:
break
if not is_excluded:
self._last = commit
return WalkEntry(self._walker, commit)
self._is_finished = True
return None
__next__ = next
class Walker(object):
"""Object for performing a walk of commits in a store.
Walker objects are initialized with a store and other options and can then
be treated as iterators of Commit objects.
"""
def __init__(self, store, include, exclude=None, order=ORDER_DATE,
reverse=False, max_entries=None, paths=None,
rename_detector=None, follow=False, since=None, until=None,
get_parents=lambda commit: commit.parents,
queue_cls=_CommitTimeQueue):
"""Constructor.
:param store: ObjectStore instance for looking up objects.
:param include: Iterable of SHAs of commits to include along with their
ancestors.
:param exclude: Iterable of SHAs of commits to exclude along with their
ancestors, overriding includes.
:param order: ORDER_* constant specifying the order of results. Anything
other than ORDER_DATE may result in O(n) memory usage.
:param reverse: If True, reverse the order of output, requiring O(n)
memory.
:param max_entries: The maximum number of entries to yield, or None for
no limit.
:param paths: Iterable of file or subtree paths to show entries for.
:param rename_detector: diff.RenameDetector object for detecting
renames.
:param follow: If True, follow path across renames/copies. Forces a
default rename_detector.
:param since: Timestamp to list commits after.
:param until: Timestamp to list commits before.
:param get_parents: Method to retrieve the parents of a commit
:param queue_cls: A class to use for a queue of commits, supporting the
iterator protocol. The constructor takes a single argument, the
Walker.
"""
# Note: when adding arguments to this method, please also update
# dulwich.repo.BaseRepo.get_walker
if order not in ALL_ORDERS:
raise ValueError('Unknown walk order %s' % order)
self.store = store
if not isinstance(include, list):
include = [include]
self.include = include
self.excluded = set(exclude or [])
self.order = order
self.reverse = reverse
self.max_entries = max_entries
self.paths = paths and set(paths) or None
if follow and not rename_detector:
rename_detector = RenameDetector(store)
self.rename_detector = rename_detector
self.get_parents = get_parents
self.follow = follow
self.since = since
self.until = until
self._num_entries = 0
self._queue = queue_cls(self)
self._out_queue = collections.deque()
def _path_matches(self, changed_path):
if changed_path is None:
return False
for followed_path in self.paths:
if changed_path == followed_path:
return True
if (changed_path.startswith(followed_path) and
changed_path[len(followed_path)] == b'/'[0]):
return True
return False
def _change_matches(self, change):
if not change:
return False
old_path = change.old.path
new_path = change.new.path
if self._path_matches(new_path):
if self.follow and change.type in RENAME_CHANGE_TYPES:
self.paths.add(old_path)
self.paths.remove(new_path)
return True
elif self._path_matches(old_path):
return True
return False
def _should_return(self, entry):
"""Determine if a walk entry should be returned..
:param entry: The WalkEntry to consider.
:return: True if the WalkEntry should be returned by this walk, or False
otherwise (e.g. if it doesn't match any requested paths).
"""
commit = entry.commit
if self.since is not None and commit.commit_time < self.since:
return False
if self.until is not None and commit.commit_time > self.until:
return False
if commit.id in self.excluded:
return False
if self.paths is None:
return True
if len(self.get_parents(commit)) > 1:
for path_changes in entry.changes():
# For merge commits, only include changes with conflicts for
# this path. Since a rename conflict may include different
# old.paths, we have to check all of them.
for change in path_changes:
if self._change_matches(change):
return True
else:
for change in entry.changes():
if self._change_matches(change):
return True
return None
def _next(self):
max_entries = self.max_entries
while max_entries is None or self._num_entries < max_entries:
entry = next(self._queue)
if entry is not None:
self._out_queue.append(entry)
if entry is None or len(self._out_queue) > _MAX_EXTRA_COMMITS:
if not self._out_queue:
return None
entry = self._out_queue.popleft()
if self._should_return(entry):
self._num_entries += 1
return entry
return None
def _reorder(self, results):
"""Possibly reorder a results iterator.
:param results: An iterator of WalkEntry objects, in the order returned
from the queue_cls.
:return: An iterator or list of WalkEntry objects, in the order required
by the Walker.
"""
if self.order == ORDER_TOPO:
results = _topo_reorder(results, self.get_parents)
if self.reverse:
results = reversed(list(results))
return results
def __iter__(self):
return iter(self._reorder(iter(self._next, None)))
def _topo_reorder(entries, get_parents=lambda commit: commit.parents):
"""Reorder an iterable of entries topologically.
This works best assuming the entries are already in almost-topological
order, e.g. in commit time order.
:param entries: An iterable of WalkEntry objects.
:param get_parents: Optional function for getting the parents of a commit.
:return: iterator over WalkEntry objects from entries in FIFO order, except
where a parent would be yielded before any of its children.
"""
todo = collections.deque()
pending = {}
num_children = defaultdict(int)
for entry in entries:
todo.append(entry)
for p in get_parents(entry.commit):
num_children[p] += 1
while todo:
entry = todo.popleft()
commit = entry.commit
commit_id = commit.id
if num_children[commit_id]:
pending[commit_id] = entry
continue
for parent_id in get_parents(commit):
num_children[parent_id] -= 1
if not num_children[parent_id]:
parent_entry = pending.pop(parent_id, None)
if parent_entry:
todo.appendleft(parent_entry)
yield entry