diff --git a/NEWS b/NEWS index af58b01b..8c689e81 100644 --- a/NEWS +++ b/NEWS @@ -1,1308 +1,1310 @@ 0.14.0 UNRELEASED BUG FIXES * Fix ShaFile.id after modification of a copied ShaFile. (Félix Mattrat, Jelmer Vernooij) IMPROVEMENTS * Add `dulwich.config.parse_submodules` function. + * Add `RefsContainer.follow` method. (#438) + 0.13.0 2016-04-24 IMPROVEMENTS * Support `ssh://` URLs in get_transport_and_path_from_url(). (Jelmer Vernooij, #402) * Support missing empty line after headers in Git commits and tags. (Nicolas Dandrimont, #413) * Fix `dulwich.porcelain.status` when used in empty trees. (Jelmer Vernooij, #415) * Return copies of objects in MemoryObjectStore rather than references, making the behaviour more consistent with that of DiskObjectStore. (Félix Mattrat, Jelmer Vernooij) * Fix ``dulwich.web`` on Python3. (#295, Jonas Haag) CHANGES * Drop support for Python 2.6. * Fix python3 client web support. (Jelmer Vernooij) BUG FIXES * Fix hang on Gzip decompression. (Jonas Haag) * Don't rely on working tell() and seek() methods on wsgi.input. (Jonas Haag) * Support fastexport/fastimport functionality on python3 with newer versions of fastimport (>= 0.9.5). (Jelmer Vernooij, Félix Mattrat) 0.12.0 2015-12-13 IMPROVEMENTS * Add a `dulwich.archive` module that can create tarballs. Based on code from Jonas Haag in klaus. * Add a `dulwich.reflog` module for reading and writing reflogs. (Jelmer Vernooij) * Fix handling of ambiguous refs in `parse_ref` to make it match the behaviour described in https://git-scm.com/docs/gitrevisions. (Chris Bunney) * Support Python3 in C modules. (Lele Gaifax) BUG FIXES * Simplify handling of SSH command invocation. Fixes quoting of paths. Thanks, Thomas Liebetraut. (#384) * Fix inconsistent handling of trailing slashes for DictRefsContainer. (#383) * Add hack to support thin packs duing fetch(), albeit while requiring the entire pack file to be loaded into memory. (jsbain) CHANGES * This will be the last release to support Python 2.6. 0.11.2 2015-09-18 IMPROVEMENTS * Add support for agent= capability. (Jelmer Vernooij, #298) * Add support for quiet capability. (Jelmer Vernooij) CHANGES * The ParamikoSSHVendor class has been moved to * dulwich.contrib.paramiko_vendor, as it's currently untested. (Jelmer Vernooij, #364) 0.11.1 2015-09-13 Fix-up release to exclude broken blame.py file. 0.11.0 2015-09-13 IMPROVEMENTS * Extended Python3 support to most of the codebase. (Gary van der Merwe, Jelmer Vernooij) * The `Repo` object has a new `close` method that can be called to close any open resources. (Gary van der Merwe) * Support 'git.bat' in SubprocessGitClient on Windows. (Stefan Zimmermann) * Advertise 'ofs-delta' capability in receive-pack server side capabilities. (Jelmer Vernooij) * Switched `default_local_git_client_cls` to `LocalGitClient`. (Gary van der Merwe) * Add `porcelain.ls_remote` and `GitClient.get_refs`. (Michael Edgar) * Add `Repo.discover` method. (B. M. Corser) * Add `dulwich.objectspec.parse_refspec`. (Jelmer Vernooij) * Add `porcelain.pack_objects` and `porcelain.repack`. (Jelmer Vernooij) BUG FIXES * Fix handling of 'done' in graph walker and implement the 'no-done' capability. (Tommy Yu, #88) * Avoid recursion limit issues resolving deltas. (William Grant, #81) * Allow arguments in local client binary path overrides. (Jelmer Vernooij) * Fix handling of commands with arguments in paramiko SSH client. (Andreas Klöckner, Jelmer Vernooij, #363) * Fix parsing of quoted strings in configs. (Jelmer Vernooij, #305) 0.10.1 2015-03-25 BUG FIXES * Return `ApplyDeltaError` when encountering delta errors in both C extensions and native delta application code. (Jelmer Vernooij, #259) 0.10.0 2015-03-22 BUG FIXES * In dulwich.index.build_index_from_tree, by default refuse to create entries that start with .git/. * Fix running of testsuite when installed. (Jelmer Vernooij, #223) * Use a block cache in _find_content_rename_candidates(), improving performance. (Mike Williams) * Add support for ``core.protectNTFS`` setting. (Jelmer Vernooij) * Fix TypeError when fetching empty updates. (Hwee Miin Koh) * Resolve delta refs when pulling into a MemoryRepo. (Max Shawabkeh, #256) * Fix handling of tags of non-commits in missing object finder. (Augie Fackler, #211) * Explicitly disable mmap on plan9 where it doesn't work. (Jeff Sickel) IMPROVEMENTS * New public method `Repo.reset_index`. (Jelmer Vernooij) * Prevent duplicate parsing of loose files in objects directory when reading. Thanks to David Keijser for the report. (Jelmer Vernooij, #231) 0.9.9 2015-03-20 SECURITY BUG FIXES * Fix buffer overflow in C implementation of pack apply_delta(). (CVE-2015-0838) Thanks to Ivan Fratric of the Google Security Team for reporting this issue. (Jelmer Vernooij) 0.9.8 2014-11-30 BUG FIXES * Various fixes to improve test suite running on Windows. (Gary van der Merwe) * Limit delta copy length to 64K in v2 pack files. (Robert Brown) * Strip newline from final ACKed SHA while fetching packs. (Michael Edgar) * Remove assignment to PyList_SIZE() that was causing segfaults on pypy. (Jelmer Vernooij, #196) IMPROVEMENTS * Add porcelain 'receive-pack' and 'upload-pack'. (Jelmer Vernooij) * Handle SIGINT signals in bin/dulwich. (Jelmer Vernooij) * Add 'status' support to bin/dulwich. (Jelmer Vernooij) * Add 'branch_create', 'branch_list', 'branch_delete' porcelain. (Jelmer Vernooij) * Add 'fetch' porcelain. (Jelmer Vernooij) * Add 'tag_delete' porcelain. (Jelmer Vernooij) * Add support for serializing/deserializing 'gpgsig' attributes in Commit. (Jelmer Vernooij) CHANGES * dul-web is now available as 'dulwich web-daemon'. (Jelmer Vernooij) * dulwich.porcelain.tag has been renamed to tag_create. dulwich.porcelain.list_tags has been renamed to tag_list. (Jelmer Vernooij) API CHANGES * Restore support for Python 2.6. (Jelmer Vernooij, Gary van der Merwe) 0.9.7 2014-06-08 BUG FIXES * Fix tests dependent on hash ordering. (Michael Edgar) * Support staging symbolic links in Repo.stage. (Robert Brown) * Ensure that all files object are closed when running the test suite. (Gary van der Merwe) * When writing OFS_DELTA pack entries, write correct offset. (Augie Fackler) * Fix handler of larger copy operations in packs. (Augie Fackler) * Various fixes to improve test suite running on Windows. (Gary van der Merwe) * Fix logic for extra adds of identical files in rename detector. (Robert Brown) IMPROVEMENTS * Add porcelain 'status'. (Ryan Faulkner) * Add porcelain 'daemon'. (Jelmer Vernooij) * Add `dulwich.greenthreads` module which provides support for concurrency of some object store operations. (Fabien Boucher) * Various changes to improve compatibility with Python 3. (Gary van der Merwe, Hannu Valtonen, michael-k) * Add OpenStack Swift backed repository implementation in dulwich.contrib. See README.swift for details. (Fabien Boucher) API CHANGES * An optional close function can be passed to the Protocol class. This will be called by its close method. (Gary van der Merwe) * All classes with close methods are now context managers, so that they can be easily closed using a `with` statement. (Gary van der Merwe) * Remove deprecated `num_objects` argument to `write_pack` methods. (Jelmer Vernooij) OTHER CHANGES * The 'dul-daemon' script has been removed. The same functionality is now available as 'dulwich daemon'. (Jelmer Vernooij) 0.9.6 2014-04-23 IMPROVEMENTS * Add support for recursive add in 'git add'. (Ryan Faulkner, Jelmer Vernooij) * Add porcelain 'list_tags'. (Ryan Faulkner) * Add porcelain 'push'. (Ryan Faulkner) * Add porcelain 'pull'. (Ryan Faulkner) * Support 'http.proxy' in HttpGitClient. (Jelmer Vernooij, #1096030) * Support 'http.useragent' in HttpGitClient. (Jelmer Vernooij) * In server, wait for clients to send empty list of wants when talking to empty repository. (Damien Tournoud) * Various changes to improve compatibility with Python 3. (Gary van der Merwe) BUG FIXES * Support unseekable 'wsgi.input' streams. (Jonas Haag) * Raise TypeError when passing unicode() object to Repo.__getitem__. (Jonas Haag) * Fix handling of `reset` command in dulwich.fastexport. (Jelmer Vernooij, #1249029) * In client, don't wait for server to close connection first. Fixes hang when used against GitHub server implementation. (Siddharth Agarwal) * DeltaChainIterator: fix a corner case where an object is inflated as an object already in the repository. (Damien Tournoud, #135) * Stop leaking file handles during pack reload. (Damien Tournoud) * Avoid reopening packs during pack cache reload. (Jelmer Vernooij) API CHANGES * Drop support for Python 2.6. (Jelmer Vernooij) 0.9.5 2014-02-23 IMPROVEMENTS * Add porcelain 'tag'. (Ryan Faulkner) * New module `dulwich.objectspec` for parsing strings referencing objects and commit ranges. (Jelmer Vernooij) * Add shallow branch support. (milki) * Allow passing urllib2 `opener` into HttpGitClient. (Dov Feldstern, #909037) CHANGES * Drop support for Python 2.4 and 2.5. (Jelmer Vernooij) API CHANGES * Remove long deprecated ``Repo.commit``, ``Repo.get_blob``, ``Repo.tree`` and ``Repo.tag``. (Jelmer Vernooij) * Remove long deprecated ``Repo.revision_history`` and ``Repo.ref``. (Jelmer Vernooij) * Remove long deprecated ``Tree.entries``. (Jelmer Vernooij) BUG FIXES * Raise KeyError rather than TypeError when passing in unicode object of length 20 or 40 to Repo.__getitem__. (Jelmer Vernooij) * Use 'rm' rather than 'unlink' in tests, since the latter does not exist on OpenBSD and other platforms. (Dmitrij D. Czarkoff) 0.9.4 2013-11-30 IMPROVEMENTS * Add ssh_kwargs attribute to ParamikoSSHVendor. (milki) * Add Repo.set_description(). (Víðir Valberg Guðmundsson) * Add a basic `dulwich.porcelain` module. (Jelmer Vernooij, Marcin Kuzminski) * Various performance improvements for object access. (Jelmer Vernooij) * New function `get_transport_and_path_from_url`, similar to `get_transport_and_path` but only supports URLs. (Jelmer Vernooij) * Add support for file:// URLs in `get_transport_and_path_from_url`. (Jelmer Vernooij) * Add LocalGitClient implementation. (Jelmer Vernooij) BUG FIXES * Support filesystems with 64bit inode and device numbers. (André Roth) CHANGES * Ref handling has been moved to dulwich.refs. (Jelmer Vernooij) API CHANGES * Remove long deprecated RefsContainer.set_ref(). (Jelmer Vernooij) * Repo.ref() is now deprecated in favour of Repo.refs[]. (Jelmer Vernooij) FEATURES * Add support for graftpoints. (milki) 0.9.3 2013-09-27 BUG FIXES * Fix path for stdint.h in MANIFEST.in. (Jelmer Vernooij) 0.9.2 2013-09-26 BUG FIXES * Include stdint.h in MANIFEST.in (Mark Mikofski) 0.9.1 2013-09-22 BUG FIXES * Support lookups of 40-character refs in BaseRepo.__getitem__. (Chow Loong Jin, Jelmer Vernooij) * Fix fetching packs with side-band-64k capability disabled. (David Keijser, Jelmer Vernooij) * Several fixes in send-pack protocol behaviour - handling of empty pack files and deletes. (milki, #1063087) * Fix capability negotiation when fetching packs over HTTP. (#1072461, William Grant) * Enforce determine_wants returning an empty list rather than None. (Fabien Boucher, Jelmer Vernooij) * In the server, support pushes just removing refs. (Fabien Boucher, Jelmer Vernooij) IMPROVEMENTS * Support passing a single revision to BaseRepo.get_walker() rather than a list of revisions. (Alberto Ruiz) * Add `Repo.get_description` method. (Jelmer Vernooij) * Support thin packs in Pack.iterobjects() and Pack.get_raw(). (William Grant) * Add `MemoryObjectStore.add_pack` and `MemoryObjectStore.add_thin_pack` methods. (David Bennett) * Add paramiko-based SSH vendor. (Aaron O'Mullan) * Support running 'dulwich.server' and 'dulwich.web' using 'python -m'. (Jelmer Vernooij) * Add ObjectStore.close(). (Jelmer Vernooij) * Raise appropriate NotImplementedError when encountering dumb HTTP servers. (Jelmer Vernooij) API CHANGES * SSHVendor.connect_ssh has been renamed to SSHVendor.run_command. (Jelmer Vernooij) * ObjectStore.add_pack() now returns a 3-tuple. The last element will be an abort() method that can be used to cancel the pack operation. (Jelmer Vernooij) 0.9.0 2013-05-31 BUG FIXES * Push efficiency - report missing objects only. (#562676, Artem Tikhomirov) * Use indentation consistent with C Git in config files. (#1031356, Curt Moore, Jelmer Vernooij) * Recognize and skip binary files in diff function. (Takeshi Kanemoto) * Fix handling of relative paths in dulwich.client.get_transport_and_path. (Brian Visel, #1169368) * Preserve ordering of entries in configuration. (Benjamin Pollack) * Support ~ expansion in SSH client paths. (milki, #1083439) * Support relative paths in alternate paths. (milki, Michel Lespinasse, #1175007) * Log all error messages from wsgiref server to the logging module. This makes the test suit quiet again. (Gary van der Merwe) * Support passing None for empty tree in changes_from_tree. (Kevin Watters) * Support fetching empty repository in client. (milki, #1060462) IMPROVEMENTS: * Add optional honor_filemode flag to build_index_from_tree. (Mark Mikofski) * Support core/filemode setting when building trees. (Jelmer Vernooij) * Add chapter on tags in tutorial. (Ryan Faulkner) FEATURES * Add support for mergetags. (milki, #963525) * Add support for posix shell hooks. (milki) 0.8.7 2012-11-27 BUG FIXES * Fix use of alternates in ``DiskObjectStore``.{__contains__,__iter__}. (Dmitriy) * Fix compatibility with Python 2.4. (David Carr) 0.8.6 2012-11-09 API CHANGES * dulwich.__init__ no longer imports client, protocol, repo and server modules. (Jelmer Vernooij) FEATURES * ConfigDict now behaves more like a dictionary. (Adam 'Cezar' Jenkins, issue #58) * HTTPGitApplication now takes an optional `fallback_app` argument. (Jonas Haag, issue #67) * Support for large pack index files. (Jameson Nash) TESTING * Make index entry tests a little bit less strict, to cope with slightly different behaviour on various platforms. (Jelmer Vernooij) * ``setup.py test`` (available when setuptools is installed) now runs all tests, not just the basic unit tests. (Jelmer Vernooij) BUG FIXES * Commit._deserialize now actually deserializes the current state rather than the previous one. (Yifan Zhang, issue #59) * Handle None elements in lists of TreeChange objects. (Alex Holmes) * Support cloning repositories without HEAD set. (D-Key, Jelmer Vernooij, issue #69) * Support ``MemoryRepo.get_config``. (Jelmer Vernooij) * In ``get_transport_and_path``, pass extra keyword arguments on to HttpGitClient. (Jelmer Vernooij) 0.8.5 2012-03-29 BUG FIXES * Avoid use of 'with' in dulwich.index. (Jelmer Vernooij) * Be a little bit strict about OS behaviour in index tests. Should fix the tests on Debian GNU/kFreeBSD. (Jelmer Vernooij) 0.8.4 2012-03-28 BUG FIXES * Options on the same line as sections in config files are now supported. (Jelmer Vernooij, #920553) * Only negotiate capabilities that are also supported by the server. (Rod Cloutier, Risto Kankkunen) * Fix parsing of invalid timezone offsets with two minus signs. (Jason R. Coombs, #697828) * Reset environment variables during tests, to avoid test isolation leaks reading ~/.gitconfig. (Risto Kankkunen) TESTS * $HOME is now explicitly specified for tests that use it to read ``~/.gitconfig``, to prevent test isolation issues. (Jelmer Vernooij, #920330) FEATURES * Additional arguments to get_transport_and_path are now passed on to the constructor of the transport. (Sam Vilain) * The WSGI server now transparently handles when a git client submits data using Content-Encoding: gzip. (David Blewett, Jelmer Vernooij) * Add dulwich.index.build_index_from_tree(). (milki) 0.8.3 2012-01-21 FEATURES * The config parser now supports the git-config file format as described in git-config(1) and can write git config files. (Jelmer Vernooij, #531092, #768687) * ``Repo.do_commit`` will now use the user identity from .git/config or ~/.gitconfig if none was explicitly specified. (Jelmer Vernooij) BUG FIXES * Allow ``determine_wants`` methods to include the zero sha in their return value. (Jelmer Vernooij) 0.8.2 2011-12-18 BUG FIXES * Cope with different zlib buffer sizes in sha1 file parser. (Jelmer Vernooij) * Fix get_transport_and_path for HTTP/HTTPS URLs. (Bruno Renié) * Avoid calling free_objects() on NULL in error cases. (Chris Eberle) * Fix use --bare argument to 'dulwich init'. (Chris Eberle) * Properly abort connections when the determine_wants function raises an exception. (Jelmer Vernooij, #856769) * Tweak xcodebuild hack to deal with more error output. (Jelmer Vernooij, #903840) FEATURES * Add support for retrieving tarballs from remote servers. (Jelmer Vernooij, #379087) * New method ``update_server_info`` which generates data for dumb server access. (Jelmer Vernooij, #731235) 0.8.1 2011-10-31 FEATURES * Repo.do_commit has a new argument 'ref'. * Repo.do_commit has a new argument 'merge_heads'. (Jelmer Vernooij) * New ``Repo.get_walker`` method. (Jelmer Vernooij) * New ``Repo.clone`` method. (Jelmer Vernooij, #725369) * ``GitClient.send_pack`` now supports the 'side-band-64k' capability. (Jelmer Vernooij) * ``HttpGitClient`` which supports the smart server protocol over HTTP. "dumb" access is not yet supported. (Jelmer Vernooij, #373688) * Add basic support for alternates. (Jelmer Vernooij, #810429) CHANGES * unittest2 or python >= 2.7 is now required for the testsuite. testtools is no longer supported. (Jelmer Vernooij, #830713) BUG FIXES * Fix compilation with older versions of MSVC. (Martin gz) * Special case 'refs/stash' as a valid ref. (Jelmer Vernooij, #695577) * Smart protocol clients can now change refs even if they are not uploading new data. (Jelmer Vernooij, #855993) * Don't compile C extensions when running in pypy. (Ronny Pfannschmidt, #881546) * Use different name for strnlen replacement function to avoid clashing with system strnlen. (Jelmer Vernooij, #880362) API CHANGES * ``Repo.revision_history`` is now deprecated in favor of ``Repo.get_walker``. (Jelmer Vernooij) 0.8.0 2011-08-07 FEATURES * New DeltaChainIterator abstract class for quickly iterating all objects in a pack, with implementations for pack indexing and inflation. (Dave Borowitz) * New walk module with a Walker class for customizable commit walking. (Dave Borowitz) * New tree_changes_for_merge function in diff_tree. (Dave Borowitz) * Easy rename detection in RenameDetector even without find_copies_harder. (Dave Borowitz) BUG FIXES * Avoid storing all objects in memory when writing pack. (Jelmer Vernooij, #813268) * Support IPv6 for git:// connections. (Jelmer Vernooij, #801543) * Improve performance of Repo.revision_history(). (Timo Schmid, #535118) * Fix use of SubprocessWrapper on Windows. (Paulo Madeira, #670035) * Fix compilation on newer versions of Mac OS X (Lion and up). (Ryan McKern, #794543) * Prevent raising ValueError for correct refs in RefContainer.__delitem__. * Correctly return a tuple from MemoryObjectStore.get_raw. (Dave Borowitz) * Fix a bug in reading the pack checksum when there are fewer than 20 bytes left in the buffer. (Dave Borowitz) * Support ~ in git:// URL paths. (Jelmer Vernooij, #813555) * Make ShaFile.__eq__ work when other is not a ShaFile. (Dave Borowitz) * ObjectStore.get_graph_walker() now no longer yields the same revision more than once. This has a significant improvement for performance when wide revision graphs are involved. (Jelmer Vernooij, #818168) * Teach ReceivePackHandler how to read empty packs. (Dave Borowitz) * Don't send a pack with duplicates of the same object. (Dave Borowitz) * Teach the server how to serve a clone of an empty repo. (Dave Borowitz) * Correctly advertise capabilities during receive-pack. (Dave Borowitz) * Fix add/add and add/rename conflicts in tree_changes_for_merge. (Dave Borowitz) * Use correct MIME types in web server. (Dave Borowitz) API CHANGES * write_pack no longer takes the num_objects argument and requires an object to be passed in that is iterable (rather than an iterator) and that provides __len__. (Jelmer Vernooij) * write_pack_data has been renamed to write_pack_objects and no longer takes a num_objects argument. (Jelmer Vernooij) * take_msb_bytes, read_zlib_chunks, unpack_objects, and PackStreamReader.read_objects now take an additional argument indicating a crc32 to compute. (Dave Borowitz) * PackObjectIterator was removed; its functionality is still exposed by PackData.iterobjects. (Dave Borowitz) * Add a sha arg to write_pack_object to incrementally compute a SHA. (Dave Borowitz) * Include offset in PackStreamReader results. (Dave Borowitz) * Move PackStreamReader from server to pack. (Dave Borowitz) * Extract a check_length_and_checksum, compute_file_sha, and pack_object_header pack helper functions. (Dave Borowitz) * Extract a compute_file_sha function. (Dave Borowitz) * Remove move_in_thin_pack as a separate method; add_thin_pack now completes the thin pack and moves it in in one step. Remove ThinPackData as well. (Dave Borowitz) * Custom buffer size in read_zlib_chunks. (Dave Borowitz) * New UnpackedObject data class that replaces ad-hoc tuples in the return value of unpack_object and various DeltaChainIterator methods. (Dave Borowitz) * Add a lookup_path convenience method to Tree. (Dave Borowitz) * Optionally create RenameDetectors without passing in tree SHAs. (Dave Borowitz) * Optionally include unchanged entries in RenameDetectors. (Dave Borowitz) * Optionally pass a RenameDetector to tree_changes. (Dave Borowitz) * Optionally pass a request object through to server handlers. (Dave Borowitz) TEST CHANGES * If setuptools is installed, "python setup.py test" will now run the testsuite. (Jelmer Vernooij) * Add a new build_pack test utility for building packs from a simple spec. (Dave Borowitz) * Add a new build_commit_graph test utility for building commits from a simple spec. (Dave Borowitz) 0.7.1 2011-04-12 BUG FIXES * Fix double decref in _diff_tree.c. (Ted Horst, #715528) * Fix the build on Windows. (Pascal Quantin) * Fix get_transport_and_path compatibility with pre-2.6.5 versions of Python. (Max Bowsher, #707438) * BaseObjectStore.determine_wants_all no longer breaks on zero SHAs. (Jelmer Vernooij) * write_tree_diff() now supports submodules. (Jelmer Vernooij) * Fix compilation for XCode 4 and older versions of distutils.sysconfig. (Daniele Sluijters) IMPROVEMENTS * Sphinxified documentation. (Lukasz Balcerzak) * Add Pack.keep.(Marc Brinkmann) API CHANGES * The order of the parameters to Tree.add(name, mode, sha) has changed, and is now consistent with the rest of Dulwich. Existing code will still work but print a DeprecationWarning. (Jelmer Vernooij, #663550) * Tree.entries() is now deprecated in favour of Tree.items() and Tree.iteritems(). (Jelmer Vernooij) 0.7.0 2011-01-21 FEATURES * New `dulwich.diff_tree` module for simple content-based rename detection. (Dave Borowitz) * Add Tree.items(). (Jelmer Vernooij) * Add eof() and unread_pkt_line() methods to Protocol. (Dave Borowitz) * Add write_tree_diff(). (Jelmer Vernooij) * Add `serve_command` function for git server commands as executables. (Jelmer Vernooij) * dulwich.client.get_transport_and_path now supports rsync-style repository URLs. (Dave Borowitz, #568493) BUG FIXES * Correct short-circuiting operation for no-op fetches in the server. (Dave Borowitz) * Support parsing git mbox patches without a version tail, as generated by Mercurial. (Jelmer Vernooij) * Fix dul-receive-pack and dul-upload-pack. (Jelmer Vernooij) * Zero-padded file modes in Tree objects no longer trigger an exception but the check code warns about them. (Augie Fackler, #581064) * Repo.init() now honors the mkdir flag. (#671159) * The ref format is now checked when setting a ref rather than when reading it back. (Dave Borowitz, #653527) * Make sure pack files are closed correctly. (Tay Ray Chuan) DOCUMENTATION * Run the tutorial inside the test suite. (Jelmer Vernooij) * Reorganized and updated the tutorial. (Jelmer Vernooij, Dave Borowitz, #610550, #610540) 0.6.2 2010-10-16 BUG FIXES * HTTP server correctly handles empty CONTENT_LENGTH. (Dave Borowitz) * Don't error when creating GitFiles with the default mode. (Dave Borowitz) * ThinPackData.from_file now works with resolve_ext_ref callback. (Dave Borowitz) * Provide strnlen() on mingw32 which doesn't have it. (Hans Kolek) * Set bare=true in the configuratin for bare repositories. (Dirk Neumann) FEATURES * Use slots for core objects to save up on memory. (Jelmer Vernooij) * Web server supports streaming progress/pack output. (Dave Borowitz) * New public function dulwich.pack.write_pack_header. (Dave Borowitz) * Distinguish between missing files and read errors in HTTP server. (Dave Borowitz) * Initial work on support for fastimport using python-fastimport. (Jelmer Vernooij) * New dulwich.pack.MemoryPackIndex class. (Jelmer Vernooij) * Delegate SHA peeling to the object store. (Dave Borowitz) TESTS * Use GitFile when modifying packed-refs in tests. (Dave Borowitz) * New tests in test_web with better coverage and fewer ad-hoc mocks. (Dave Borowitz) * Standardize quote delimiters in test_protocol. (Dave Borowitz) * Fix use when testtools is installed. (Jelmer Vernooij) * Add trivial test for write_pack_header. (Jelmer Vernooij) * Refactor some of dulwich.tests.compat.server_utils. (Dave Borowitz) * Allow overwriting id property of objects in test utils. (Dave Borowitz) * Use real in-memory objects rather than stubs for server tests. (Dave Borowitz) * Clean up MissingObjectFinder. (Dave Borowitz) API CHANGES * ObjectStore.iter_tree_contents now walks contents in depth-first, sorted order. (Dave Borowitz) * ObjectStore.iter_tree_contents can optionally yield tree objects as well. (Dave Borowitz). * Add side-band-64k support to ReceivePackHandler. (Dave Borowitz) * Change server capabilities methods to classmethods. (Dave Borowitz) * Tweak server handler injection. (Dave Borowitz) * PackIndex1 and PackIndex2 now subclass FilePackIndex, which is itself a subclass of PackIndex. (Jelmer Vernooij) DOCUMENTATION * Add docstrings for various functions in dulwich.objects. (Jelmer Vernooij) * Clean up docstrings in dulwich.protocol. (Dave Borowitz) * Explicitly specify allowed protocol commands to ProtocolGraphWalker.read_proto_line. (Dave Borowitz) * Add utility functions to DictRefsContainer. (Dave Borowitz) 0.6.1 2010-07-22 BUG FIXES * Fix memory leak in C implementation of sorted_tree_items. (Dave Borowitz) * Use correct path separators for named repo files. (Dave Borowitz) * python > 2.7 and testtools-based test runners will now also pick up skipped tests correctly. (Jelmer Vernooij) FEATURES * Move named file initilization to BaseRepo. (Dave Borowitz) * Add logging utilities and git/HTTP server logging. (Dave Borowitz) * The GitClient interface has been cleaned up and instances are now reusable. (Augie Fackler) * Allow overriding paths to executables in GitSSHClient. (Ross Light, Jelmer Vernooij, #585204) * Add PackBasedObjectStore.pack_loose_objects(). (Jelmer Vernooij) TESTS * Add tests for sorted_tree_items and C implementation. (Dave Borowitz) * Add a MemoryRepo that stores everything in memory. (Dave Borowitz) * Quiet logging output from web tests. (Dave Borowitz) * More flexible version checking for compat tests. (Dave Borowitz) * Compat tests for servers with and without side-band-64k. (Dave Borowitz) CLEANUP * Clean up file headers. (Dave Borowitz) TESTS * Use GitFile when modifying packed-refs in tests. (Dave Borowitz) API CHANGES * dulwich.pack.write_pack_index_v{1,2} now take a file-like object rather than a filename. (Jelmer Vernooij) * Make dul-daemon/dul-web trivial wrappers around server functionality. (Dave Borowitz) * Move reference WSGI handler to web.py. (Dave Borowitz) * Factor out _report_status in ReceivePackHandler. (Dave Borowitz) * Factor out a function to convert a line to a pkt-line. (Dave Borowitz) 0.6.0 2010-05-22 note: This list is most likely incomplete for 0.6.0. BUG FIXES * Fix ReceivePackHandler to disallow removing refs without delete-refs. (Dave Borowitz) * Deal with capabilities required by the client, even if they can not be disabled in the server. (Dave Borowitz) * Fix trailing newlines in generated patch files. (Jelmer Vernooij) * Implement RefsContainer.__contains__. (Jelmer Vernooij) * Cope with \r in ref files on Windows. ( http://github.com/jelmer/dulwich/issues/#issue/13, Jelmer Vernooij) * Fix GitFile breakage on Windows. (Anatoly Techtonik, #557585) * Support packed ref deletion with no peeled refs. (Augie Fackler) * Fix send pack when there is nothing to fetch. (Augie Fackler) * Fix fetch if no progress function is specified. (Augie Fackler) * Allow double-staging of files that are deleted in the index. (Dave Borowitz) * Fix RefsContainer.add_if_new to support dangling symrefs. (Dave Borowitz) * Non-existant index files in non-bare repositories are now treated as empty. (Dave Borowitz) * Always update ShaFile.id when the contents of the object get changed. (Jelmer Vernooij) * Various Python2.4-compatibility fixes. (Dave Borowitz) * Fix thin pack handling. (Dave Borowitz) FEATURES * Add include-tag capability to server. (Dave Borowitz) * New dulwich.fastexport module that can generate fastexport streams. (Jelmer Vernooij) * Implemented BaseRepo.__contains__. (Jelmer Vernooij) * Add __setitem__ to DictRefsContainer. (Dave Borowitz) * Overall improvements checking Git objects. (Dave Borowitz) * Packs are now verified while they are received. (Dave Borowitz) TESTS * Add framework for testing compatibility with C Git. (Dave Borowitz) * Add various tests for the use of non-bare repositories. (Dave Borowitz) * Cope with diffstat not being available on all platforms. (Tay Ray Chuan, Jelmer Vernooij) * Add make_object and make_commit convenience functions to test utils. (Dave Borowitz) API BREAKAGES * The 'committer' and 'message' arguments to Repo.do_commit() have been swapped. 'committer' is now optional. (Jelmer Vernooij) * Repo.get_blob, Repo.commit, Repo.tag and Repo.tree are now deprecated. (Jelmer Vernooij) * RefsContainer.set_ref() was renamed to RefsContainer.set_symbolic_ref(), for clarity. (Jelmer Vernooij) API CHANGES * The primary serialization APIs in dulwich.objects now work with chunks of strings rather than with full-text strings. (Jelmer Vernooij) 0.5.02010-03-03 BUG FIXES * Support custom fields in commits (readonly). (Jelmer Vernooij) * Improved ref handling. (Dave Borowitz) * Rework server protocol to be smarter and interoperate with cgit client. (Dave Borowitz) * Add a GitFile class that uses the same locking protocol for writes as cgit. (Dave Borowitz) * Cope with forward slashes correctly in the index on Windows. (Jelmer Vernooij, #526793) FEATURES * --pure option to setup.py to allow building/installing without the C extensions. (Hal Wine, Anatoly Techtonik, Jelmer Vernooij, #434326) * Implement Repo.get_config(). (Jelmer Vernooij, Augie Fackler) * HTTP dumb and smart server. (Dave Borowitz) * Add abstract baseclass for Repo that does not require file system operations. (Dave Borowitz) 0.4.1 2010-01-03 FEATURES * Add ObjectStore.iter_tree_contents(). (Jelmer Vernooij) * Add Index.changes_from_tree(). (Jelmer Vernooij) * Add ObjectStore.tree_changes(). (Jelmer Vernooij) * Add functionality for writing patches in dulwich.patch. (Jelmer Vernooij) 0.4.0 2009-10-07 DOCUMENTATION * Added tutorial. API CHANGES * dulwich.object_store.tree_lookup_path will now return the mode and sha of the object found rather than the object itself. BUG FIXES * Use binascii.hexlify / binascii.unhexlify for better performance. * Cope with extra unknown data in index files by ignoring it (for now). * Add proper error message when server unexpectedly hangs up. (#415843) * Correctly write opcode for equal in create_delta. 0.3.3 2009-07-23 FEATURES * Implement ShaFile.__hash__(). * Implement Tree.__len__() BUG FIXES * Check for 'objects' and 'refs' directories when looking for a Git repository. (#380818) 0.3.2 2009-05-20 BUG FIXES * Support the encoding field in Commits. * Some Windows compatibility fixes. * Fixed several issues in commit support. FEATURES * Basic support for handling submodules. 0.3.1 2009-05-13 FEATURES * Implemented Repo.__getitem__, Repo.__setitem__ and Repo.__delitem__ to access content. API CHANGES * Removed Repo.set_ref, Repo.remove_ref, Repo.tags, Repo.get_refs and Repo.heads in favor of Repo.refs, a dictionary-like object for accessing refs. BUG FIXES * Removed import of 'sha' module in objects.py, which was causing deprecation warnings on Python 2.6. 0.3.0 2009-05-10 FEATURES * A new function 'commit_tree' has been added that can commit a tree based on an index. BUG FIXES * The memory usage when generating indexes has been significantly reduced. * A memory leak in the C implementation of parse_tree has been fixed. * The send-pack smart server command now works. (Thanks Scott Chacon) * The handling of short timestamps (less than 10 digits) has been fixed. * The handling of timezones has been fixed. 0.2.1 2009-04-30 BUG FIXES * Fix compatibility with Python2.4. 0.2.0 2009-04-30 FEATURES * Support for activity reporting in smart protocol client. * Optional C extensions for better performance in a couple of places that are performance-critical. 0.1.1 2009-03-13 BUG FIXES * Fixed regression in Repo.find_missing_objects() * Don't fetch ^{} objects from remote hosts, as requesting them causes a hangup. * Always write pack to disk completely before calculating checksum. FEATURES * Allow disabling thin packs when talking to remote hosts. 0.1.0 2009-01-24 * Initial release. diff --git a/dulwich/refs.py b/dulwich/refs.py index df3a32e2..49e06876 100644 --- a/dulwich/refs.py +++ b/dulwich/refs.py @@ -1,762 +1,767 @@ # refs.py -- For dealing with git refs # Copyright (C) 2008-2013 Jelmer Vernooij # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; version 2 # of the License or (at your option) any later version of # the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. """Ref handling. """ import errno import os import sys from dulwich.errors import ( PackedRefsException, RefFormatError, ) from dulwich.objects import ( git_line, valid_hexsha, ) from dulwich.file import ( GitFile, ensure_dir_exists, ) SYMREF = b'ref: ' LOCAL_BRANCH_PREFIX = b'refs/heads/' BAD_REF_CHARS = set(b'\177 ~^:?*[') def check_ref_format(refname): """Check if a refname is correctly formatted. Implements all the same rules as git-check-ref-format[1]. [1] http://www.kernel.org/pub/software/scm/git/docs/git-check-ref-format.html :param refname: The refname to check :return: True if refname is valid, False otherwise """ # These could be combined into one big expression, but are listed separately # to parallel [1]. if b'/.' in refname or refname.startswith(b'.'): return False if b'/' not in refname: return False if b'..' in refname: return False for i, c in enumerate(refname): if ord(refname[i:i+1]) < 0o40 or c in BAD_REF_CHARS: return False if refname[-1] in b'/.': return False if refname.endswith(b'.lock'): return False if b'@{' in refname: return False if b'\\' in refname: return False return True class RefsContainer(object): """A container for refs.""" def set_symbolic_ref(self, name, other): """Make a ref point at another ref. :param name: Name of the ref to set :param other: Name of the ref to point at """ raise NotImplementedError(self.set_symbolic_ref) def get_packed_refs(self): """Get contents of the packed-refs file. :return: Dictionary mapping ref names to SHA1s :note: Will return an empty dictionary when no packed-refs file is present. """ raise NotImplementedError(self.get_packed_refs) def get_peeled(self, name): """Return the cached peeled value of a ref, if available. :param name: Name of the ref to peel :return: The peeled value of the ref. If the ref is known not point to a tag, this will be the SHA the ref refers to. If the ref may point to a tag, but no cached information is available, None is returned. """ return None def import_refs(self, base, other): for name, value in other.items(): self[b'/'.join((base, name))] = value def allkeys(self): """All refs present in this container.""" raise NotImplementedError(self.allkeys) def keys(self, base=None): """Refs present in this container. :param base: An optional base to return refs under. :return: An unsorted set of valid refs in this container, including packed refs. """ if base is not None: return self.subkeys(base) else: return self.allkeys() def subkeys(self, base): """Refs present in this container under a base. :param base: The base to return refs under. :return: A set of valid refs in this container under the base; the base prefix is stripped from the ref names returned. """ keys = set() base_len = len(base) + 1 for refname in self.allkeys(): if refname.startswith(base): keys.add(refname[base_len:]) return keys def as_dict(self, base=None): """Return the contents of this container as a dictionary. """ ret = {} keys = self.keys(base) if base is None: base = b'' else: base = base.rstrip(b'/') for key in keys: try: ret[key] = self[(base + b'/' + key).strip(b'/')] except KeyError: continue # Unable to resolve return ret def _check_refname(self, name): """Ensure a refname is valid and lives in refs or is HEAD. HEAD is not a valid refname according to git-check-ref-format, but this class needs to be able to touch HEAD. Also, check_ref_format expects refnames without the leading 'refs/', but this class requires that so it cannot touch anything outside the refs dir (or HEAD). :param name: The name of the reference. :raises KeyError: if a refname is not HEAD or is otherwise not valid. """ if name in (b'HEAD', b'refs/stash'): return if not name.startswith(b'refs/') or not check_ref_format(name[5:]): raise RefFormatError(name) def read_ref(self, refname): """Read a reference without following any references. :param refname: The name of the reference :return: The contents of the ref file, or None if it does not exist. """ contents = self.read_loose_ref(refname) if not contents: contents = self.get_packed_refs().get(refname, None) return contents def read_loose_ref(self, name): """Read a loose reference and return its contents. :param name: the refname to read :return: The contents of the ref file, or None if it does not exist. """ raise NotImplementedError(self.read_loose_ref) - def _follow(self, name): + def follow(self, name): """Follow a reference name. - :return: a tuple of (refname, sha), where refname is the name of the - last reference in the symbolic reference chain + :return: a tuple of (refnames, sha), wheres refnames are the names of + references in the chain """ contents = SYMREF + name depth = 0 + refnames = [] while contents.startswith(SYMREF): refname = contents[len(SYMREF):] + refnames.append(refname) contents = self.read_ref(refname) if not contents: break depth += 1 if depth > 5: raise KeyError(name) - return refname, contents + return refnames, contents def __contains__(self, refname): if self.read_ref(refname): return True return False def __getitem__(self, name): """Get the SHA1 for a reference name. This method follows all symbolic references. """ - _, sha = self._follow(name) + _, sha = self.follow(name) if sha is None: raise KeyError(name) return sha def set_if_equals(self, name, old_ref, new_ref): """Set a refname to new_ref only if it currently equals old_ref. This method follows all symbolic references if applicable for the subclass, and can be used to perform an atomic compare-and-swap operation. :param name: The refname to set. :param old_ref: The old sha the refname must refer to, or None to set unconditionally. :param new_ref: The new sha the refname will refer to. :return: True if the set was successful, False otherwise. """ raise NotImplementedError(self.set_if_equals) def add_if_new(self, name, ref): """Add a new reference only if it does not already exist.""" raise NotImplementedError(self.add_if_new) def __setitem__(self, name, ref): """Set a reference name to point to the given SHA1. This method follows all symbolic references if applicable for the subclass. :note: This method unconditionally overwrites the contents of a reference. To update atomically only if the reference has not changed, use set_if_equals(). :param name: The refname to set. :param ref: The new sha the refname will refer to. """ self.set_if_equals(name, None, ref) def remove_if_equals(self, name, old_ref): """Remove a refname only if it currently equals old_ref. This method does not follow symbolic references, even if applicable for the subclass. It can be used to perform an atomic compare-and-delete operation. :param name: The refname to delete. :param old_ref: The old sha the refname must refer to, or None to delete unconditionally. :return: True if the delete was successful, False otherwise. """ raise NotImplementedError(self.remove_if_equals) def __delitem__(self, name): """Remove a refname. This method does not follow symbolic references, even if applicable for the subclass. :note: This method unconditionally deletes the contents of a reference. To delete atomically only if the reference has not changed, use remove_if_equals(). :param name: The refname to delete. """ self.remove_if_equals(name, None) class DictRefsContainer(RefsContainer): """RefsContainer backed by a simple dict. This container does not support symbolic or packed references and is not threadsafe. """ def __init__(self, refs): self._refs = refs self._peeled = {} def allkeys(self): return self._refs.keys() def read_loose_ref(self, name): return self._refs.get(name, None) def get_packed_refs(self): return {} def set_symbolic_ref(self, name, other): self._refs[name] = SYMREF + other def set_if_equals(self, name, old_ref, new_ref): if old_ref is not None and self._refs.get(name, None) != old_ref: return False - realname, _ = self._follow(name) - self._check_refname(realname) - self._refs[realname] = new_ref + realnames, _ = self.follow(name) + for realname in realnames: + self._check_refname(realname) + self._refs[realname] = new_ref return True def add_if_new(self, name, ref): if name in self._refs: return False self._refs[name] = ref return True def remove_if_equals(self, name, old_ref): if old_ref is not None and self._refs.get(name, None) != old_ref: return False del self._refs[name] return True def get_peeled(self, name): return self._peeled.get(name) def _update(self, refs): """Update multiple refs; intended only for testing.""" # TODO(dborowitz): replace this with a public function that uses # set_if_equal. self._refs.update(refs) def _update_peeled(self, peeled): """Update cached peeled refs; intended only for testing.""" self._peeled.update(peeled) class InfoRefsContainer(RefsContainer): """Refs container that reads refs from a info/refs file.""" def __init__(self, f): self._refs = {} self._peeled = {} for l in f.readlines(): sha, name = l.rstrip(b'\n').split(b'\t') if name.endswith(b'^{}'): name = name[:-3] if not check_ref_format(name): raise ValueError("invalid ref name %r" % name) self._peeled[name] = sha else: if not check_ref_format(name): raise ValueError("invalid ref name %r" % name) self._refs[name] = sha def allkeys(self): return self._refs.keys() def read_loose_ref(self, name): return self._refs.get(name, None) def get_packed_refs(self): return {} def get_peeled(self, name): try: return self._peeled[name] except KeyError: return self._refs[name] class DiskRefsContainer(RefsContainer): """Refs container that reads refs from disk.""" def __init__(self, path): self.path = path self._packed_refs = None self._peeled_refs = None def __repr__(self): return "%s(%r)" % (self.__class__.__name__, self.path) def subkeys(self, base): subkeys = set() path = self.refpath(base) for root, dirs, files in os.walk(path): dir = root[len(path):].strip(os.path.sep).replace(os.path.sep, "/") for filename in files: refname = (("%s/%s" % (dir, filename)) .strip("/").encode(sys.getfilesystemencoding())) # check_ref_format requires at least one /, so we prepend the # base before calling it. if check_ref_format(base + b'/' + refname): subkeys.add(refname) for key in self.get_packed_refs(): if key.startswith(base): subkeys.add(key[len(base):].strip(b'/')) return subkeys def allkeys(self): allkeys = set() if os.path.exists(self.refpath(b'HEAD')): allkeys.add(b'HEAD') path = self.refpath(b'') for root, dirs, files in os.walk(self.refpath(b'refs')): dir = root[len(path):].strip(os.path.sep).replace(os.path.sep, "/") for filename in files: refname = ("%s/%s" % (dir, filename)).encode(sys.getfilesystemencoding()) if check_ref_format(refname): allkeys.add(refname) allkeys.update(self.get_packed_refs()) return allkeys def refpath(self, name): """Return the disk path of a ref. """ if getattr(self.path, "encode", None) and getattr(name, "decode", None): name = name.decode(sys.getfilesystemencoding()) if os.path.sep != "/": name = name.replace("/", os.path.sep) return os.path.join(self.path, name) def get_packed_refs(self): """Get contents of the packed-refs file. :return: Dictionary mapping ref names to SHA1s :note: Will return an empty dictionary when no packed-refs file is present. """ # TODO: invalidate the cache on repacking if self._packed_refs is None: # set both to empty because we want _peeled_refs to be # None if and only if _packed_refs is also None. self._packed_refs = {} self._peeled_refs = {} path = os.path.join(self.path, 'packed-refs') try: f = GitFile(path, 'rb') except IOError as e: if e.errno == errno.ENOENT: return {} raise with f: first_line = next(iter(f)).rstrip() if (first_line.startswith(b'# pack-refs') and b' peeled' in first_line): for sha, name, peeled in read_packed_refs_with_peeled(f): self._packed_refs[name] = sha if peeled: self._peeled_refs[name] = peeled else: f.seek(0) for sha, name in read_packed_refs(f): self._packed_refs[name] = sha return self._packed_refs def get_peeled(self, name): """Return the cached peeled value of a ref, if available. :param name: Name of the ref to peel :return: The peeled value of the ref. If the ref is known not point to a tag, this will be the SHA the ref refers to. If the ref may point to a tag, but no cached information is available, None is returned. """ self.get_packed_refs() if self._peeled_refs is None or name not in self._packed_refs: # No cache: no peeled refs were read, or this ref is loose return None if name in self._peeled_refs: return self._peeled_refs[name] else: # Known not peelable return self[name] def read_loose_ref(self, name): """Read a reference file and return its contents. If the reference file a symbolic reference, only read the first line of the file. Otherwise, only read the first 40 bytes. :param name: the refname to read, relative to refpath :return: The contents of the ref file, or None if the file does not exist. :raises IOError: if any other error occurs """ filename = self.refpath(name) try: with GitFile(filename, 'rb') as f: header = f.read(len(SYMREF)) if header == SYMREF: # Read only the first line return header + next(iter(f)).rstrip(b'\r\n') else: # Read only the first 40 bytes return header + f.read(40 - len(SYMREF)) except IOError as e: if e.errno == errno.ENOENT: return None raise def _remove_packed_ref(self, name): if self._packed_refs is None: return filename = os.path.join(self.path, 'packed-refs') # reread cached refs from disk, while holding the lock f = GitFile(filename, 'wb') try: self._packed_refs = None self.get_packed_refs() if name not in self._packed_refs: return del self._packed_refs[name] if name in self._peeled_refs: del self._peeled_refs[name] write_packed_refs(f, self._packed_refs, self._peeled_refs) f.close() finally: f.abort() def set_symbolic_ref(self, name, other): """Make a ref point at another ref. :param name: Name of the ref to set :param other: Name of the ref to point at """ self._check_refname(name) self._check_refname(other) filename = self.refpath(name) try: f = GitFile(filename, 'wb') try: f.write(SYMREF + other + b'\n') except (IOError, OSError): f.abort() raise finally: f.close() def set_if_equals(self, name, old_ref, new_ref): """Set a refname to new_ref only if it currently equals old_ref. This method follows all symbolic references, and can be used to perform an atomic compare-and-swap operation. :param name: The refname to set. :param old_ref: The old sha the refname must refer to, or None to set unconditionally. :param new_ref: The new sha the refname will refer to. :return: True if the set was successful, False otherwise. """ self._check_refname(name) try: - realname, _ = self._follow(name) - except KeyError: + realnames, _ = self.follow(name) + realname = realnames[-1] + except (KeyError, IndexError): realname = name filename = self.refpath(realname) ensure_dir_exists(os.path.dirname(filename)) with GitFile(filename, 'wb') as f: if old_ref is not None: try: # read again while holding the lock orig_ref = self.read_loose_ref(realname) if orig_ref is None: orig_ref = self.get_packed_refs().get(realname, None) if orig_ref != old_ref: f.abort() return False except (OSError, IOError): f.abort() raise try: f.write(new_ref + b'\n') except (OSError, IOError): f.abort() raise return True def add_if_new(self, name, ref): """Add a new reference only if it does not already exist. This method follows symrefs, and only ensures that the last ref in the chain does not exist. :param name: The refname to set. :param ref: The new sha the refname will refer to. :return: True if the add was successful, False otherwise. """ try: - realname, contents = self._follow(name) + realnames, contents = self.follow(name) if contents is not None: return False - except KeyError: + realname = realnames[-1] + except (KeyError, IndexError): realname = name self._check_refname(realname) filename = self.refpath(realname) ensure_dir_exists(os.path.dirname(filename)) with GitFile(filename, 'wb') as f: if os.path.exists(filename) or name in self.get_packed_refs(): f.abort() return False try: f.write(ref + b'\n') except (OSError, IOError): f.abort() raise return True def remove_if_equals(self, name, old_ref): """Remove a refname only if it currently equals old_ref. This method does not follow symbolic references. It can be used to perform an atomic compare-and-delete operation. :param name: The refname to delete. :param old_ref: The old sha the refname must refer to, or None to delete unconditionally. :return: True if the delete was successful, False otherwise. """ self._check_refname(name) filename = self.refpath(name) ensure_dir_exists(os.path.dirname(filename)) f = GitFile(filename, 'wb') try: if old_ref is not None: orig_ref = self.read_loose_ref(name) if orig_ref is None: orig_ref = self.get_packed_refs().get(name, None) if orig_ref != old_ref: return False # may only be packed try: os.remove(filename) except OSError as e: if e.errno != errno.ENOENT: raise self._remove_packed_ref(name) finally: # never write, we just wanted the lock f.abort() return True def _split_ref_line(line): """Split a single ref line into a tuple of SHA1 and name.""" fields = line.rstrip(b'\n').split(b' ') if len(fields) != 2: raise PackedRefsException("invalid ref line %r" % line) sha, name = fields if not valid_hexsha(sha): raise PackedRefsException("Invalid hex sha %r" % sha) if not check_ref_format(name): raise PackedRefsException("invalid ref name %r" % name) return (sha, name) def read_packed_refs(f): """Read a packed refs file. :param f: file-like object to read from :return: Iterator over tuples with SHA1s and ref names. """ for l in f: if l.startswith(b'#'): # Comment continue if l.startswith(b'^'): raise PackedRefsException( "found peeled ref in packed-refs without peeled") yield _split_ref_line(l) def read_packed_refs_with_peeled(f): """Read a packed refs file including peeled refs. Assumes the "# pack-refs with: peeled" line was already read. Yields tuples with ref names, SHA1s, and peeled SHA1s (or None). :param f: file-like object to read from, seek'ed to the second line """ last = None for l in f: if l[0] == b'#': continue l = l.rstrip(b'\r\n') if l.startswith(b'^'): if not last: raise PackedRefsException("unexpected peeled ref line") if not valid_hexsha(l[1:]): raise PackedRefsException("Invalid hex sha %r" % l[1:]) sha, name = _split_ref_line(last) last = None yield (sha, name, l[1:]) else: if last: sha, name = _split_ref_line(last) yield (sha, name, None) last = l if last: sha, name = _split_ref_line(last) yield (sha, name, None) def write_packed_refs(f, packed_refs, peeled_refs=None): """Write a packed refs file. :param f: empty file-like object to write to :param packed_refs: dict of refname to sha of packed refs to write :param peeled_refs: dict of refname to peeled value of sha """ if peeled_refs is None: peeled_refs = {} else: f.write(b'# pack-refs with: peeled\n') for refname in sorted(packed_refs.keys()): f.write(git_line(packed_refs[refname], refname)) if refname in peeled_refs: f.write(b'^' + peeled_refs[refname] + b'\n') def read_info_refs(f): ret = {} for l in f.readlines(): (sha, name) = l.rstrip("\r\n").split("\t", 1) ret[name] = sha return ret def write_info_refs(refs, store): """Generate info refs.""" for name, sha in sorted(refs.items()): # get_refs() includes HEAD as a special case, but we don't want to # advertise it if name == b'HEAD': continue try: o = store[sha] except KeyError: continue peeled = store.peel_sha(sha) yield o.id + b'\t' + name + b'\n' if o.id != peeled.id: yield peeled.id + b'\t' + name + b'^{}\n' is_local_branch = lambda x: x.startswith(b'refs/heads/') diff --git a/dulwich/repo.py b/dulwich/repo.py index 1b7a325f..6d9589bf 100644 --- a/dulwich/repo.py +++ b/dulwich/repo.py @@ -1,1026 +1,1026 @@ # repo.py -- For dealing with git repositories. # Copyright (C) 2007 James Westby # Copyright (C) 2008-2013 Jelmer Vernooij # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; version 2 # of the License or (at your option) any later version of # the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. """Repository access. This module contains the base class for git repositories (BaseRepo) and an implementation which uses a repository on local disk (Repo). """ from io import BytesIO import errno import os import sys from dulwich.errors import ( NoIndexPresent, NotBlobError, NotCommitError, NotGitRepository, NotTreeError, NotTagError, CommitError, RefFormatError, HookError, ) from dulwich.file import ( GitFile, ) from dulwich.object_store import ( DiskObjectStore, MemoryObjectStore, ObjectStoreGraphWalker, ) from dulwich.objects import ( check_hexsha, Blob, Commit, ShaFile, Tag, Tree, ) from dulwich.hooks import ( PreCommitShellHook, PostCommitShellHook, CommitMsgShellHook, ) from dulwich.refs import ( check_ref_format, RefsContainer, DictRefsContainer, InfoRefsContainer, DiskRefsContainer, read_packed_refs, read_packed_refs_with_peeled, write_packed_refs, SYMREF, ) import warnings CONTROLDIR = '.git' OBJECTDIR = 'objects' REFSDIR = 'refs' REFSDIR_TAGS = 'tags' REFSDIR_HEADS = 'heads' INDEX_FILENAME = "index" BASE_DIRECTORIES = [ ["branches"], [REFSDIR], [REFSDIR, REFSDIR_TAGS], [REFSDIR, REFSDIR_HEADS], ["hooks"], ["info"] ] def parse_graftpoints(graftpoints): """Convert a list of graftpoints into a dict :param graftpoints: Iterator of graftpoint lines Each line is formatted as: []* Resulting dictionary is: : [*] https://git.wiki.kernel.org/index.php/GraftPoint """ grafts = {} for l in graftpoints: raw_graft = l.split(None, 1) commit = raw_graft[0] if len(raw_graft) == 2: parents = raw_graft[1].split() else: parents = [] for sha in [commit] + parents: check_hexsha(sha, 'Invalid graftpoint') grafts[commit] = parents return grafts def serialize_graftpoints(graftpoints): """Convert a dictionary of grafts into string The graft dictionary is: : [*] Each line is formatted as: []* https://git.wiki.kernel.org/index.php/GraftPoint """ graft_lines = [] for commit, parents in graftpoints.items(): if parents: graft_lines.append(commit + b' ' + b' '.join(parents)) else: graft_lines.append(commit) return b'\n'.join(graft_lines) class BaseRepo(object): """Base class for a git repository. :ivar object_store: Dictionary-like object for accessing the objects :ivar refs: Dictionary-like object with the refs in this repository """ def __init__(self, object_store, refs): """Open a repository. This shouldn't be called directly, but rather through one of the base classes, such as MemoryRepo or Repo. :param object_store: Object store to use :param refs: Refs container to use """ self.object_store = object_store self.refs = refs self._graftpoints = {} self.hooks = {} def _init_files(self, bare): """Initialize a default set of named files.""" from dulwich.config import ConfigFile self._put_named_file('description', b"Unnamed repository") f = BytesIO() cf = ConfigFile() cf.set(b"core", b"repositoryformatversion", b"0") cf.set(b"core", b"filemode", b"true") cf.set(b"core", b"bare", bare) cf.set(b"core", b"logallrefupdates", True) cf.write_to_file(f) self._put_named_file('config', f.getvalue()) self._put_named_file(os.path.join('info', 'exclude'), b'') def get_named_file(self, path): """Get a file from the control dir with a specific name. Although the filename should be interpreted as a filename relative to the control dir in a disk-based Repo, the object returned need not be pointing to a file in that location. :param path: The path to the file, relative to the control dir. :return: An open file object, or None if the file does not exist. """ raise NotImplementedError(self.get_named_file) def _put_named_file(self, path, contents): """Write a file to the control dir with the given name and contents. :param path: The path to the file, relative to the control dir. :param contents: A string to write to the file. """ raise NotImplementedError(self._put_named_file) def open_index(self): """Open the index for this repository. :raise NoIndexPresent: If no index is present :return: The matching `Index` """ raise NotImplementedError(self.open_index) def fetch(self, target, determine_wants=None, progress=None): """Fetch objects into another repository. :param target: The target repository :param determine_wants: Optional function to determine what refs to fetch. :param progress: Optional progress function :return: The local refs """ if determine_wants is None: determine_wants = target.object_store.determine_wants_all target.object_store.add_objects( self.fetch_objects(determine_wants, target.get_graph_walker(), progress)) return self.get_refs() def fetch_objects(self, determine_wants, graph_walker, progress, get_tagged=None): """Fetch the missing objects required for a set of revisions. :param determine_wants: Function that takes a dictionary with heads and returns the list of heads to fetch. :param graph_walker: Object that can iterate over the list of revisions to fetch and has an "ack" method that will be called to acknowledge that a revision is present. :param progress: Simple progress function that will be called with updated progress strings. :param get_tagged: Function that returns a dict of pointed-to sha -> tag sha for including tags. :return: iterator over objects, with __len__ implemented """ wants = determine_wants(self.get_refs()) if not isinstance(wants, list): raise TypeError("determine_wants() did not return a list") shallows = getattr(graph_walker, 'shallow', frozenset()) unshallows = getattr(graph_walker, 'unshallow', frozenset()) if wants == []: # TODO(dborowitz): find a way to short-circuit that doesn't change # this interface. if shallows or unshallows: # Do not send a pack in shallow short-circuit path return None return [] # If the graph walker is set up with an implementation that can # ACK/NAK to the wire, it will write data to the client through # this call as a side-effect. haves = self.object_store.find_common_revisions(graph_walker) # Deal with shallow requests separately because the haves do # not reflect what objects are missing if shallows or unshallows: haves = [] # TODO: filter the haves commits from iter_shas. # the specific commits aren't missing. def get_parents(commit): if commit.id in shallows: return [] return self.get_parents(commit.id, commit) return self.object_store.iter_shas( self.object_store.find_missing_objects( haves, wants, progress, get_tagged, get_parents=get_parents)) def get_graph_walker(self, heads=None): """Retrieve a graph walker. A graph walker is used by a remote repository (or proxy) to find out which objects are present in this repository. :param heads: Repository heads to use (optional) :return: A graph walker object """ if heads is None: heads = self.refs.as_dict(b'refs/heads').values() return ObjectStoreGraphWalker(heads, self.get_parents) def get_refs(self): """Get dictionary with all refs. :return: A ``dict`` mapping ref names to SHA1s """ return self.refs.as_dict() def head(self): """Return the SHA1 pointed at by HEAD.""" return self.refs[b'HEAD'] def _get_object(self, sha, cls): assert len(sha) in (20, 40) ret = self.get_object(sha) if not isinstance(ret, cls): if cls is Commit: raise NotCommitError(ret) elif cls is Blob: raise NotBlobError(ret) elif cls is Tree: raise NotTreeError(ret) elif cls is Tag: raise NotTagError(ret) else: raise Exception("Type invalid: %r != %r" % ( ret.type_name, cls.type_name)) return ret def get_object(self, sha): """Retrieve the object with the specified SHA. :param sha: SHA to retrieve :return: A ShaFile object :raise KeyError: when the object can not be found """ return self.object_store[sha] def get_parents(self, sha, commit=None): """Retrieve the parents of a specific commit. If the specific commit is a graftpoint, the graft parents will be returned instead. :param sha: SHA of the commit for which to retrieve the parents :param commit: Optional commit matching the sha :return: List of parents """ try: return self._graftpoints[sha] except KeyError: if commit is None: commit = self[sha] return commit.parents def get_config(self): """Retrieve the config object. :return: `ConfigFile` object for the ``.git/config`` file. """ raise NotImplementedError(self.get_config) def get_description(self): """Retrieve the description for this repository. :return: String with the description of the repository as set by the user. """ raise NotImplementedError(self.get_description) def set_description(self, description): """Set the description for this repository. :param description: Text to set as description for this repository. """ raise NotImplementedError(self.set_description) def get_config_stack(self): """Return a config stack for this repository. This stack accesses the configuration for both this repository itself (.git/config) and the global configuration, which usually lives in ~/.gitconfig. :return: `Config` instance for this repository """ from dulwich.config import StackedConfig backends = [self.get_config()] + StackedConfig.default_backends() return StackedConfig(backends, writable=backends[0]) def get_peeled(self, ref): """Get the peeled value of a ref. :param ref: The refname to peel. :return: The fully-peeled SHA1 of a tag object, after peeling all intermediate tags; if the original ref does not point to a tag, this will equal the original SHA1. """ cached = self.refs.get_peeled(ref) if cached is not None: return cached return self.object_store.peel_sha(self.refs[ref]).id def get_walker(self, include=None, *args, **kwargs): """Obtain a walker for this repository. :param include: Iterable of SHAs of commits to include along with their ancestors. Defaults to [HEAD] :param exclude: Iterable of SHAs of commits to exclude along with their ancestors, overriding includes. :param order: ORDER_* constant specifying the order of results. Anything other than ORDER_DATE may result in O(n) memory usage. :param reverse: If True, reverse the order of output, requiring O(n) memory. :param max_entries: The maximum number of entries to yield, or None for no limit. :param paths: Iterable of file or subtree paths to show entries for. :param rename_detector: diff.RenameDetector object for detecting renames. :param follow: If True, follow path across renames/copies. Forces a default rename_detector. :param since: Timestamp to list commits after. :param until: Timestamp to list commits before. :param queue_cls: A class to use for a queue of commits, supporting the iterator protocol. The constructor takes a single argument, the Walker. :return: A `Walker` object """ from dulwich.walk import Walker if include is None: include = [self.head()] if isinstance(include, str): include = [include] kwargs['get_parents'] = lambda commit: self.get_parents(commit.id, commit) return Walker(self.object_store, include, *args, **kwargs) def __getitem__(self, name): """Retrieve a Git object by SHA1 or ref. :param name: A Git object SHA1 or a ref name :return: A `ShaFile` object, such as a Commit or Blob :raise KeyError: when the specified ref or object does not exist """ if not isinstance(name, bytes): raise TypeError("'name' must be bytestring, not %.80s" % type(name).__name__) if len(name) in (20, 40): try: return self.object_store[name] except (KeyError, ValueError): pass try: return self.object_store[self.refs[name]] except RefFormatError: raise KeyError(name) def __contains__(self, name): """Check if a specific Git object or ref is present. :param name: Git object SHA1 or ref name """ if len(name) in (20, 40): return name in self.object_store or name in self.refs else: return name in self.refs def __setitem__(self, name, value): """Set a ref. :param name: ref name :param value: Ref value - either a ShaFile object, or a hex sha """ if name.startswith(b"refs/") or name == b'HEAD': if isinstance(value, ShaFile): self.refs[name] = value.id elif isinstance(value, bytes): self.refs[name] = value else: raise TypeError(value) else: raise ValueError(name) def __delitem__(self, name): """Remove a ref. :param name: Name of the ref to remove """ if name.startswith(b"refs/") or name == b"HEAD": del self.refs[name] else: raise ValueError(name) def _get_user_identity(self): """Determine the identity to use for new commits. """ config = self.get_config_stack() return (config.get((b"user", ), b"name") + b" <" + config.get((b"user", ), b"email") + b">") def _add_graftpoints(self, updated_graftpoints): """Add or modify graftpoints :param updated_graftpoints: Dict of commit shas to list of parent shas """ # Simple validation for commit, parents in updated_graftpoints.items(): for sha in [commit] + parents: check_hexsha(sha, 'Invalid graftpoint') self._graftpoints.update(updated_graftpoints) def _remove_graftpoints(self, to_remove=[]): """Remove graftpoints :param to_remove: List of commit shas """ for sha in to_remove: del self._graftpoints[sha] def do_commit(self, message=None, committer=None, author=None, commit_timestamp=None, commit_timezone=None, author_timestamp=None, author_timezone=None, tree=None, encoding=None, ref=b'HEAD', merge_heads=None): """Create a new commit. :param message: Commit message :param committer: Committer fullname :param author: Author fullname (defaults to committer) :param commit_timestamp: Commit timestamp (defaults to now) :param commit_timezone: Commit timestamp timezone (defaults to GMT) :param author_timestamp: Author timestamp (defaults to commit timestamp) :param author_timezone: Author timestamp timezone (defaults to commit timestamp timezone) :param tree: SHA1 of the tree root to use (if not specified the current index will be committed). :param encoding: Encoding :param ref: Optional ref to commit to (defaults to current branch) :param merge_heads: Merge heads (defaults to .git/MERGE_HEADS) :return: New commit SHA1 """ import time c = Commit() if tree is None: index = self.open_index() c.tree = index.commit(self.object_store) else: if len(tree) != 40: raise ValueError("tree must be a 40-byte hex sha string") c.tree = tree try: self.hooks['pre-commit'].execute() except HookError as e: raise CommitError(e) except KeyError: # no hook defined, silent fallthrough pass if merge_heads is None: # FIXME: Read merge heads from .git/MERGE_HEADS merge_heads = [] if committer is None: # FIXME: Support GIT_COMMITTER_NAME/GIT_COMMITTER_EMAIL environment # variables committer = self._get_user_identity() c.committer = committer if commit_timestamp is None: # FIXME: Support GIT_COMMITTER_DATE environment variable commit_timestamp = time.time() c.commit_time = int(commit_timestamp) if commit_timezone is None: # FIXME: Use current user timezone rather than UTC commit_timezone = 0 c.commit_timezone = commit_timezone if author is None: # FIXME: Support GIT_AUTHOR_NAME/GIT_AUTHOR_EMAIL environment # variables author = committer c.author = author if author_timestamp is None: # FIXME: Support GIT_AUTHOR_DATE environment variable author_timestamp = commit_timestamp c.author_time = int(author_timestamp) if author_timezone is None: author_timezone = commit_timezone c.author_timezone = author_timezone if encoding is not None: c.encoding = encoding if message is None: # FIXME: Try to read commit message from .git/MERGE_MSG raise ValueError("No commit message specified") try: c.message = self.hooks['commit-msg'].execute(message) if c.message is None: c.message = message except HookError as e: raise CommitError(e) except KeyError: # no hook defined, message not modified c.message = message if ref is None: # Create a dangling commit c.parents = merge_heads self.object_store.add_object(c) else: try: old_head = self.refs[ref] c.parents = [old_head] + merge_heads self.object_store.add_object(c) ok = self.refs.set_if_equals(ref, old_head, c.id) except KeyError: c.parents = merge_heads self.object_store.add_object(c) ok = self.refs.add_if_new(ref, c.id) if not ok: # Fail if the atomic compare-and-swap failed, leaving the commit and # all its objects as garbage. raise CommitError("%s changed during commit" % (ref,)) try: self.hooks['post-commit'].execute() except HookError as e: # silent failure warnings.warn("post-commit hook failed: %s" % e, UserWarning) except KeyError: # no hook defined, silent fallthrough pass return c.id def read_gitfile(f): """Read a ``.git`` file. The first line of the file should start with "gitdir: " :param f: File-like object to read from :return: A path """ cs = f.read() if not cs.startswith("gitdir: "): raise ValueError("Expected file to start with 'gitdir: '") return cs[len("gitdir: "):].rstrip("\n") class Repo(BaseRepo): """A git repository backed by local disk. To open an existing repository, call the contructor with the path of the repository. To create a new repository, use the Repo.init class method. """ def __init__(self, root): hidden_path = os.path.join(root, CONTROLDIR) if os.path.isdir(os.path.join(hidden_path, OBJECTDIR)): self.bare = False self._controldir = hidden_path elif (os.path.isdir(os.path.join(root, OBJECTDIR)) and os.path.isdir(os.path.join(root, REFSDIR))): self.bare = True self._controldir = root elif os.path.isfile(hidden_path): self.bare = False with open(hidden_path, 'r') as f: path = read_gitfile(f) self.bare = False self._controldir = os.path.join(root, path) else: raise NotGitRepository( "No git repository was found at %(path)s" % dict(path=root) ) self.path = root object_store = DiskObjectStore(os.path.join(self.controldir(), OBJECTDIR)) refs = DiskRefsContainer(self.controldir()) BaseRepo.__init__(self, object_store, refs) self._graftpoints = {} graft_file = self.get_named_file(os.path.join("info", "grafts")) if graft_file: with graft_file: self._graftpoints.update(parse_graftpoints(graft_file)) graft_file = self.get_named_file("shallow") if graft_file: with graft_file: self._graftpoints.update(parse_graftpoints(graft_file)) self.hooks['pre-commit'] = PreCommitShellHook(self.controldir()) self.hooks['commit-msg'] = CommitMsgShellHook(self.controldir()) self.hooks['post-commit'] = PostCommitShellHook(self.controldir()) @classmethod def discover(cls, start='.'): """Iterate parent directories to discover a repository Return a Repo object for the first parent directory that looks like a Git repository. :param start: The directory to start discovery from (defaults to '.') """ remaining = True path = os.path.abspath(start) while remaining: try: return cls(path) except NotGitRepository: path, remaining = os.path.split(path) raise NotGitRepository( "No git repository was found at %(path)s" % dict(path=start) ) def controldir(self): """Return the path of the control directory.""" return self._controldir def _put_named_file(self, path, contents): """Write a file to the control dir with the given name and contents. :param path: The path to the file, relative to the control dir. :param contents: A string to write to the file. """ path = path.lstrip(os.path.sep) with GitFile(os.path.join(self.controldir(), path), 'wb') as f: f.write(contents) def get_named_file(self, path): """Get a file from the control dir with a specific name. Although the filename should be interpreted as a filename relative to the control dir in a disk-based Repo, the object returned need not be pointing to a file in that location. :param path: The path to the file, relative to the control dir. :return: An open file object, or None if the file does not exist. """ # TODO(dborowitz): sanitize filenames, since this is used directly by # the dumb web serving code. path = path.lstrip(os.path.sep) try: return open(os.path.join(self.controldir(), path), 'rb') except (IOError, OSError) as e: if e.errno == errno.ENOENT: return None raise def index_path(self): """Return path to the index file.""" return os.path.join(self.controldir(), INDEX_FILENAME) def open_index(self): """Open the index for this repository. :raise NoIndexPresent: If no index is present :return: The matching `Index` """ from dulwich.index import Index if not self.has_index(): raise NoIndexPresent() return Index(self.index_path()) def has_index(self): """Check if an index is present.""" # Bare repos must never have index files; non-bare repos may have a # missing index file, which is treated as empty. return not self.bare def stage(self, fs_paths): """Stage a set of paths. :param fs_paths: List of paths, relative to the repository path """ root_path_bytes = self.path.encode(sys.getfilesystemencoding()) if not isinstance(fs_paths, list): fs_paths = [fs_paths] from dulwich.index import ( blob_from_path_and_stat, index_entry_from_stat, _fs_to_tree_path, ) index = self.open_index() for fs_path in fs_paths: if not isinstance(fs_path, bytes): fs_path = fs_path.encode(sys.getfilesystemencoding()) tree_path = _fs_to_tree_path(fs_path) full_path = os.path.join(root_path_bytes, fs_path) try: st = os.lstat(full_path) except OSError: # File no longer exists try: del index[tree_path] except KeyError: pass # already removed else: blob = blob_from_path_and_stat(full_path, st) self.object_store.add_object(blob) index[tree_path] = index_entry_from_stat(st, blob.id, 0) index.write() def clone(self, target_path, mkdir=True, bare=False, origin=b"origin"): """Clone this repository. :param target_path: Target path :param mkdir: Create the target directory :param bare: Whether to create a bare repository :param origin: Base name for refs in target repository cloned from this repository :return: Created repository as `Repo` """ if not bare: target = self.init(target_path, mkdir=mkdir) else: target = self.init_bare(target_path) self.fetch(target) target.refs.import_refs( b'refs/remotes/' + origin, self.refs.as_dict(b'refs/heads')) target.refs.import_refs( b'refs/tags', self.refs.as_dict(b'refs/tags')) try: target.refs.add_if_new( b'refs/heads/master', self.refs[b'refs/heads/master']) except KeyError: pass # Update target head - head, head_sha = self.refs._follow(b'HEAD') - if head is not None and head_sha is not None: - target.refs.set_symbolic_ref(b'HEAD', head) + head_chain, head_sha = self.refs.follow(b'HEAD') + if head_chain and head_sha is not None: + target.refs.set_symbolic_ref(b'HEAD', head_chain[-1]) target[b'HEAD'] = head_sha if not bare: # Checkout HEAD to target dir target.reset_index() return target def reset_index(self, tree=None): """Reset the index back to a specific tree. :param tree: Tree SHA to reset to, None for current HEAD tree. """ from dulwich.index import ( build_index_from_tree, validate_path_element_default, validate_path_element_ntfs, ) if tree is None: tree = self[b'HEAD'].tree config = self.get_config() honor_filemode = config.get_boolean('core', 'filemode', os.name != "nt") if config.get_boolean('core', 'core.protectNTFS', os.name == "nt"): validate_path_element = validate_path_element_ntfs else: validate_path_element = validate_path_element_default return build_index_from_tree(self.path, self.index_path(), self.object_store, tree, honor_filemode=honor_filemode, validate_path_element=validate_path_element) def get_config(self): """Retrieve the config object. :return: `ConfigFile` object for the ``.git/config`` file. """ from dulwich.config import ConfigFile path = os.path.join(self._controldir, 'config') try: return ConfigFile.from_path(path) except (IOError, OSError) as e: if e.errno != errno.ENOENT: raise ret = ConfigFile() ret.path = path return ret def get_description(self): """Retrieve the description of this repository. :return: A string describing the repository or None. """ path = os.path.join(self._controldir, 'description') try: with GitFile(path, 'rb') as f: return f.read() except (IOError, OSError) as e: if e.errno != errno.ENOENT: raise return None def __repr__(self): return "" % self.path def set_description(self, description): """Set the description for this repository. :param description: Text to set as description for this repository. """ self._put_named_file('description', description) @classmethod def _init_maybe_bare(cls, path, bare): for d in BASE_DIRECTORIES: os.mkdir(os.path.join(path, *d)) DiskObjectStore.init(os.path.join(path, OBJECTDIR)) ret = cls(path) ret.refs.set_symbolic_ref(b'HEAD', b"refs/heads/master") ret._init_files(bare) return ret @classmethod def init(cls, path, mkdir=False): """Create a new repository. :param path: Path in which to create the repository :param mkdir: Whether to create the directory :return: `Repo` instance """ if mkdir: os.mkdir(path) controldir = os.path.join(path, CONTROLDIR) os.mkdir(controldir) cls._init_maybe_bare(controldir, False) return cls(path) @classmethod def init_bare(cls, path): """Create a new bare repository. ``path`` should already exist and be an emty directory. :param path: Path to create bare repository in :return: a `Repo` instance """ return cls._init_maybe_bare(path, True) create = init_bare def close(self): """Close any files opened by this repository.""" self.object_store.close() class MemoryRepo(BaseRepo): """Repo that stores refs, objects, and named files in memory. MemoryRepos are always bare: they have no working tree and no index, since those have a stronger dependency on the filesystem. """ def __init__(self): from dulwich.config import ConfigFile BaseRepo.__init__(self, MemoryObjectStore(), DictRefsContainer({})) self._named_files = {} self.bare = True self._config = ConfigFile() def _put_named_file(self, path, contents): """Write a file to the control dir with the given name and contents. :param path: The path to the file, relative to the control dir. :param contents: A string to write to the file. """ self._named_files[path] = contents def get_named_file(self, path): """Get a file from the control dir with a specific name. Although the filename should be interpreted as a filename relative to the control dir in a disk-baked Repo, the object returned need not be pointing to a file in that location. :param path: The path to the file, relative to the control dir. :return: An open file object, or None if the file does not exist. """ contents = self._named_files.get(path, None) if contents is None: return None return BytesIO(contents) def open_index(self): """Fail to open index for this repo, since it is bare. :raise NoIndexPresent: Raised when no index is present """ raise NoIndexPresent() def get_config(self): """Retrieve the config object. :return: `ConfigFile` object. """ return self._config def get_description(self): """Retrieve the repository description. This defaults to None, for no description. """ return None @classmethod def init_bare(cls, objects, refs): """Create a new bare repository in memory. :param objects: Objects for the new repository, as iterable :param refs: Refs as dictionary, mapping names to object SHA1s """ ret = cls() for obj in objects: ret.object_store.add_object(obj) for refname, sha in refs.items(): ret.refs[refname] = sha ret._init_files(bare=True) return ret diff --git a/dulwich/tests/test_refs.py b/dulwich/tests/test_refs.py index 61547a1d..d0eaf9ec 100644 --- a/dulwich/tests/test_refs.py +++ b/dulwich/tests/test_refs.py @@ -1,509 +1,509 @@ # test_refs.py -- tests for refs.py # encoding: utf-8 # Copyright (C) 2013 Jelmer Vernooij # # This program is free software; you can redistribute it and/or # modify it under the terms of the GNU General Public License # as published by the Free Software Foundation; version 2 # of the License or (at your option) any later version of # the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, # MA 02110-1301, USA. """Tests for dulwich.refs.""" from io import BytesIO import os import sys import tempfile from dulwich import errors from dulwich.file import ( GitFile, ) from dulwich.refs import ( DictRefsContainer, InfoRefsContainer, check_ref_format, _split_ref_line, read_packed_refs_with_peeled, read_packed_refs, write_packed_refs, ) from dulwich.repo import Repo from dulwich.tests import ( SkipTest, TestCase, ) from dulwich.tests.utils import ( open_repo, tear_down_repo, ) class CheckRefFormatTests(TestCase): """Tests for the check_ref_format function. These are the same tests as in the git test suite. """ def test_valid(self): self.assertTrue(check_ref_format(b'heads/foo')) self.assertTrue(check_ref_format(b'foo/bar/baz')) self.assertTrue(check_ref_format(b'refs///heads/foo')) self.assertTrue(check_ref_format(b'foo./bar')) self.assertTrue(check_ref_format(b'heads/foo@bar')) self.assertTrue(check_ref_format(b'heads/fix.lock.error')) def test_invalid(self): self.assertFalse(check_ref_format(b'foo')) self.assertFalse(check_ref_format(b'heads/foo/')) self.assertFalse(check_ref_format(b'./foo')) self.assertFalse(check_ref_format(b'.refs/foo')) self.assertFalse(check_ref_format(b'heads/foo..bar')) self.assertFalse(check_ref_format(b'heads/foo?bar')) self.assertFalse(check_ref_format(b'heads/foo.lock')) self.assertFalse(check_ref_format(b'heads/v@{ation')) self.assertFalse(check_ref_format(b'heads/foo\bar')) ONES = b'1' * 40 TWOS = b'2' * 40 THREES = b'3' * 40 FOURS = b'4' * 40 class PackedRefsFileTests(TestCase): def test_split_ref_line_errors(self): self.assertRaises(errors.PackedRefsException, _split_ref_line, b'singlefield') self.assertRaises(errors.PackedRefsException, _split_ref_line, b'badsha name') self.assertRaises(errors.PackedRefsException, _split_ref_line, ONES + b' bad/../refname') def test_read_without_peeled(self): f = BytesIO(b'\n'.join([ b'# comment', ONES + b' ref/1', TWOS + b' ref/2'])) self.assertEqual([(ONES, b'ref/1'), (TWOS, b'ref/2')], list(read_packed_refs(f))) def test_read_without_peeled_errors(self): f = BytesIO(b'\n'.join([ ONES + b' ref/1', b'^' + TWOS])) self.assertRaises(errors.PackedRefsException, list, read_packed_refs(f)) def test_read_with_peeled(self): f = BytesIO(b'\n'.join([ ONES + b' ref/1', TWOS + b' ref/2', b'^' + THREES, FOURS + b' ref/4'])) self.assertEqual([ (ONES, b'ref/1', None), (TWOS, b'ref/2', THREES), (FOURS, b'ref/4', None), ], list(read_packed_refs_with_peeled(f))) def test_read_with_peeled_errors(self): f = BytesIO(b'\n'.join([ b'^' + TWOS, ONES + b' ref/1'])) self.assertRaises(errors.PackedRefsException, list, read_packed_refs(f)) f = BytesIO(b'\n'.join([ ONES + b' ref/1', b'^' + TWOS, b'^' + THREES])) self.assertRaises(errors.PackedRefsException, list, read_packed_refs(f)) def test_write_with_peeled(self): f = BytesIO() write_packed_refs(f, {b'ref/1': ONES, b'ref/2': TWOS}, {b'ref/1': THREES}) self.assertEqual( b'\n'.join([b'# pack-refs with: peeled', ONES + b' ref/1', b'^' + THREES, TWOS + b' ref/2']) + b'\n', f.getvalue()) def test_write_without_peeled(self): f = BytesIO() write_packed_refs(f, {b'ref/1': ONES, b'ref/2': TWOS}) self.assertEqual(b'\n'.join([ONES + b' ref/1', TWOS + b' ref/2']) + b'\n', f.getvalue()) # Dict of refs that we expect all RefsContainerTests subclasses to define. _TEST_REFS = { b'HEAD': b'42d06bd4b77fed026b154d16493e5deab78f02ec', b'refs/heads/40-char-ref-aaaaaaaaaaaaaaaaaa': b'42d06bd4b77fed026b154d16493e5deab78f02ec', b'refs/heads/master': b'42d06bd4b77fed026b154d16493e5deab78f02ec', b'refs/heads/packed': b'42d06bd4b77fed026b154d16493e5deab78f02ec', b'refs/tags/refs-0.1': b'df6800012397fb85c56e7418dd4eb9405dee075c', b'refs/tags/refs-0.2': b'3ec9c43c84ff242e3ef4a9fc5bc111fd780a76a8', } class RefsContainerTests(object): def test_keys(self): actual_keys = set(self._refs.keys()) self.assertEqual(set(self._refs.allkeys()), actual_keys) # ignore the symref loop if it exists actual_keys.discard(b'refs/heads/loop') self.assertEqual(set(_TEST_REFS.keys()), actual_keys) actual_keys = self._refs.keys(b'refs/heads') actual_keys.discard(b'loop') self.assertEqual( [b'40-char-ref-aaaaaaaaaaaaaaaaaa', b'master', b'packed'], sorted(actual_keys)) self.assertEqual([b'refs-0.1', b'refs-0.2'], sorted(self._refs.keys(b'refs/tags'))) def test_as_dict(self): # refs/heads/loop does not show up even if it exists self.assertEqual(_TEST_REFS, self._refs.as_dict()) def test_setitem(self): self._refs[b'refs/some/ref'] = b'42d06bd4b77fed026b154d16493e5deab78f02ec' self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'refs/some/ref']) self.assertRaises( errors.RefFormatError, self._refs.__setitem__, b'notrefs/foo', b'42d06bd4b77fed026b154d16493e5deab78f02ec') def test_set_if_equals(self): nines = b'9' * 40 self.assertFalse(self._refs.set_if_equals(b'HEAD', b'c0ffee', nines)) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'HEAD']) self.assertTrue(self._refs.set_if_equals( b'HEAD', b'42d06bd4b77fed026b154d16493e5deab78f02ec', nines)) self.assertEqual(nines, self._refs[b'HEAD']) self.assertTrue(self._refs.set_if_equals(b'refs/heads/master', None, nines)) self.assertEqual(nines, self._refs[b'refs/heads/master']) def test_add_if_new(self): nines = b'9' * 40 self.assertFalse(self._refs.add_if_new(b'refs/heads/master', nines)) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'refs/heads/master']) self.assertTrue(self._refs.add_if_new(b'refs/some/ref', nines)) self.assertEqual(nines, self._refs[b'refs/some/ref']) def test_set_symbolic_ref(self): self._refs.set_symbolic_ref(b'refs/heads/symbolic', b'refs/heads/master') self.assertEqual(b'ref: refs/heads/master', self._refs.read_loose_ref(b'refs/heads/symbolic')) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'refs/heads/symbolic']) def test_set_symbolic_ref_overwrite(self): nines = b'9' * 40 self.assertFalse(b'refs/heads/symbolic' in self._refs) self._refs[b'refs/heads/symbolic'] = nines self.assertEqual(nines, self._refs.read_loose_ref(b'refs/heads/symbolic')) self._refs.set_symbolic_ref(b'refs/heads/symbolic', b'refs/heads/master') self.assertEqual(b'ref: refs/heads/master', self._refs.read_loose_ref(b'refs/heads/symbolic')) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'refs/heads/symbolic']) def test_check_refname(self): self._refs._check_refname(b'HEAD') self._refs._check_refname(b'refs/stash') self._refs._check_refname(b'refs/heads/foo') self.assertRaises(errors.RefFormatError, self._refs._check_refname, b'refs') self.assertRaises(errors.RefFormatError, self._refs._check_refname, b'notrefs/foo') def test_contains(self): self.assertTrue(b'refs/heads/master' in self._refs) self.assertFalse(b'refs/heads/bar' in self._refs) def test_delitem(self): self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'refs/heads/master']) del self._refs[b'refs/heads/master'] self.assertRaises(KeyError, lambda: self._refs[b'refs/heads/master']) def test_remove_if_equals(self): self.assertFalse(self._refs.remove_if_equals(b'HEAD', b'c0ffee')) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'HEAD']) self.assertTrue(self._refs.remove_if_equals( b'refs/tags/refs-0.2', b'3ec9c43c84ff242e3ef4a9fc5bc111fd780a76a8')) self.assertFalse(b'refs/tags/refs-0.2' in self._refs) class DictRefsContainerTests(RefsContainerTests, TestCase): def setUp(self): TestCase.setUp(self) self._refs = DictRefsContainer(dict(_TEST_REFS)) def test_invalid_refname(self): # FIXME: Move this test into RefsContainerTests, but requires # some way of injecting invalid refs. self._refs._refs[b'refs/stash'] = b'00' * 20 expected_refs = dict(_TEST_REFS) expected_refs[b'refs/stash'] = b'00' * 20 self.assertEqual(expected_refs, self._refs.as_dict()) class DiskRefsContainerTests(RefsContainerTests, TestCase): def setUp(self): TestCase.setUp(self) self._repo = open_repo('refs.git') self.addCleanup(tear_down_repo, self._repo) self._refs = self._repo.refs def test_get_packed_refs(self): self.assertEqual({ b'refs/heads/packed': b'42d06bd4b77fed026b154d16493e5deab78f02ec', b'refs/tags/refs-0.1': b'df6800012397fb85c56e7418dd4eb9405dee075c', }, self._refs.get_packed_refs()) def test_get_peeled_not_packed(self): # not packed self.assertEqual(None, self._refs.get_peeled(b'refs/tags/refs-0.2')) self.assertEqual(b'3ec9c43c84ff242e3ef4a9fc5bc111fd780a76a8', self._refs[b'refs/tags/refs-0.2']) # packed, known not peelable self.assertEqual(self._refs[b'refs/heads/packed'], self._refs.get_peeled(b'refs/heads/packed')) # packed, peeled self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs.get_peeled(b'refs/tags/refs-0.1')) def test_setitem(self): RefsContainerTests.test_setitem(self) f = open(os.path.join(self._refs.path, 'refs', 'some', 'ref'), 'rb') self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', f.read()[:40]) f.close() def test_setitem_symbolic(self): ones = b'1' * 40 self._refs[b'HEAD'] = ones self.assertEqual(ones, self._refs[b'HEAD']) # ensure HEAD was not modified f = open(os.path.join(self._refs.path, 'HEAD'), 'rb') self.assertEqual(b'ref: refs/heads/master', next(iter(f)).rstrip(b'\n')) f.close() # ensure the symbolic link was written through f = open(os.path.join(self._refs.path, 'refs', 'heads', 'master'), 'rb') self.assertEqual(ones, f.read()[:40]) f.close() def test_set_if_equals(self): RefsContainerTests.test_set_if_equals(self) # ensure symref was followed self.assertEqual(b'9' * 40, self._refs[b'refs/heads/master']) # ensure lockfile was deleted self.assertFalse(os.path.exists( os.path.join(self._refs.path, 'refs', 'heads', 'master.lock'))) self.assertFalse(os.path.exists( os.path.join(self._refs.path, 'HEAD.lock'))) def test_add_if_new_packed(self): # don't overwrite packed ref self.assertFalse(self._refs.add_if_new(b'refs/tags/refs-0.1', b'9' * 40)) self.assertEqual(b'df6800012397fb85c56e7418dd4eb9405dee075c', self._refs[b'refs/tags/refs-0.1']) def test_add_if_new_symbolic(self): # Use an empty repo instead of the default. repo_dir = os.path.join(tempfile.mkdtemp(), 'test') os.makedirs(repo_dir) repo = Repo.init(repo_dir) self.addCleanup(tear_down_repo, repo) refs = repo.refs nines = b'9' * 40 self.assertEqual(b'ref: refs/heads/master', refs.read_ref(b'HEAD')) self.assertFalse(b'refs/heads/master' in refs) self.assertTrue(refs.add_if_new(b'HEAD', nines)) self.assertEqual(b'ref: refs/heads/master', refs.read_ref(b'HEAD')) self.assertEqual(nines, refs[b'HEAD']) self.assertEqual(nines, refs[b'refs/heads/master']) self.assertFalse(refs.add_if_new(b'HEAD', b'1' * 40)) self.assertEqual(nines, refs[b'HEAD']) self.assertEqual(nines, refs[b'refs/heads/master']) def test_follow(self): - self.assertEqual((b'refs/heads/master', + self.assertEqual(([b'HEAD', b'refs/heads/master'], b'42d06bd4b77fed026b154d16493e5deab78f02ec'), - self._refs._follow(b'HEAD')) - self.assertEqual((b'refs/heads/master', + self._refs.follow(b'HEAD')) + self.assertEqual(([b'refs/heads/master'], b'42d06bd4b77fed026b154d16493e5deab78f02ec'), - self._refs._follow(b'refs/heads/master')) - self.assertRaises(KeyError, self._refs._follow, b'refs/heads/loop') + self._refs.follow(b'refs/heads/master')) + self.assertRaises(KeyError, self._refs.follow, b'refs/heads/loop') def test_delitem(self): RefsContainerTests.test_delitem(self) ref_file = os.path.join(self._refs.path, 'refs', 'heads', 'master') self.assertFalse(os.path.exists(ref_file)) self.assertFalse(b'refs/heads/master' in self._refs.get_packed_refs()) def test_delitem_symbolic(self): self.assertEqual(b'ref: refs/heads/master', self._refs.read_loose_ref(b'HEAD')) del self._refs[b'HEAD'] self.assertRaises(KeyError, lambda: self._refs[b'HEAD']) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs[b'refs/heads/master']) self.assertFalse(os.path.exists(os.path.join(self._refs.path, 'HEAD'))) def test_remove_if_equals_symref(self): # HEAD is a symref, so shouldn't equal its dereferenced value self.assertFalse(self._refs.remove_if_equals( b'HEAD', b'42d06bd4b77fed026b154d16493e5deab78f02ec')) self.assertTrue(self._refs.remove_if_equals( b'refs/heads/master', b'42d06bd4b77fed026b154d16493e5deab78f02ec')) self.assertRaises(KeyError, lambda: self._refs[b'refs/heads/master']) # HEAD is now a broken symref self.assertRaises(KeyError, lambda: self._refs[b'HEAD']) self.assertEqual(b'ref: refs/heads/master', self._refs.read_loose_ref(b'HEAD')) self.assertFalse(os.path.exists( os.path.join(self._refs.path, 'refs', 'heads', 'master.lock'))) self.assertFalse(os.path.exists( os.path.join(self._refs.path, 'HEAD.lock'))) def test_remove_packed_without_peeled(self): refs_file = os.path.join(self._repo.path, 'packed-refs') f = GitFile(refs_file) refs_data = f.read() f.close() f = GitFile(refs_file, 'wb') f.write(b'\n'.join(l for l in refs_data.split(b'\n') if not l or l[0] not in b'#^')) f.close() self._repo = Repo(self._repo.path) refs = self._repo.refs self.assertTrue(refs.remove_if_equals( b'refs/heads/packed', b'42d06bd4b77fed026b154d16493e5deab78f02ec')) def test_remove_if_equals_packed(self): # test removing ref that is only packed self.assertEqual(b'df6800012397fb85c56e7418dd4eb9405dee075c', self._refs[b'refs/tags/refs-0.1']) self.assertTrue( self._refs.remove_if_equals( b'refs/tags/refs-0.1', b'df6800012397fb85c56e7418dd4eb9405dee075c')) self.assertRaises(KeyError, lambda: self._refs[b'refs/tags/refs-0.1']) def test_read_ref(self): self.assertEqual(b'ref: refs/heads/master', self._refs.read_ref(b'HEAD')) self.assertEqual(b'42d06bd4b77fed026b154d16493e5deab78f02ec', self._refs.read_ref(b'refs/heads/packed')) self.assertEqual(None, self._refs.read_ref(b'nonexistant')) def test_non_ascii(self): try: encoded_ref = u'refs/tags/schön'.encode(sys.getfilesystemencoding()) except UnicodeEncodeError: raise SkipTest("filesystem encoding doesn't support special character") p = os.path.join(self._repo.path, 'refs', 'tags', 'schön') with open(p, 'w') as f: f.write('00' * 20) expected_refs = dict(_TEST_REFS) expected_refs[encoded_ref] = b'00' * 20 self.assertEqual(expected_refs, self._repo.get_refs()) _TEST_REFS_SERIALIZED = ( b'42d06bd4b77fed026b154d16493e5deab78f02ec\trefs/heads/40-char-ref-aaaaaaaaaaaaaaaaaa\n' b'42d06bd4b77fed026b154d16493e5deab78f02ec\trefs/heads/master\n' b'42d06bd4b77fed026b154d16493e5deab78f02ec\trefs/heads/packed\n' b'df6800012397fb85c56e7418dd4eb9405dee075c\trefs/tags/refs-0.1\n' b'3ec9c43c84ff242e3ef4a9fc5bc111fd780a76a8\trefs/tags/refs-0.2\n') class InfoRefsContainerTests(TestCase): def test_invalid_refname(self): text = _TEST_REFS_SERIALIZED + b'00' * 20 + b'\trefs/stash\n' refs = InfoRefsContainer(BytesIO(text)) expected_refs = dict(_TEST_REFS) del expected_refs[b'HEAD'] expected_refs[b'refs/stash'] = b'00' * 20 self.assertEqual(expected_refs, refs.as_dict()) def test_keys(self): refs = InfoRefsContainer(BytesIO(_TEST_REFS_SERIALIZED)) actual_keys = set(refs.keys()) self.assertEqual(set(refs.allkeys()), actual_keys) # ignore the symref loop if it exists actual_keys.discard(b'refs/heads/loop') expected_refs = dict(_TEST_REFS) del expected_refs[b'HEAD'] self.assertEqual(set(expected_refs.keys()), actual_keys) actual_keys = refs.keys(b'refs/heads') actual_keys.discard(b'loop') self.assertEqual( [b'40-char-ref-aaaaaaaaaaaaaaaaaa', b'master', b'packed'], sorted(actual_keys)) self.assertEqual([b'refs-0.1', b'refs-0.2'], sorted(refs.keys(b'refs/tags'))) def test_as_dict(self): refs = InfoRefsContainer(BytesIO(_TEST_REFS_SERIALIZED)) # refs/heads/loop does not show up even if it exists expected_refs = dict(_TEST_REFS) del expected_refs[b'HEAD'] self.assertEqual(expected_refs, refs.as_dict()) def test_contains(self): refs = InfoRefsContainer(BytesIO(_TEST_REFS_SERIALIZED)) self.assertTrue(b'refs/heads/master' in refs) self.assertFalse(b'refs/heads/bar' in refs) def test_get_peeled(self): refs = InfoRefsContainer(BytesIO(_TEST_REFS_SERIALIZED)) # refs/heads/loop does not show up even if it exists self.assertEqual( _TEST_REFS[b'refs/heads/master'], refs.get_peeled(b'refs/heads/master'))