Page MenuHomeSoftware Heritage
Create Task
Maniphest T822

Gitorious import: ObjectFormatException raised when badly formatted object (around date?)
Closed, MigratedEdits Locked
Actions

Description

Reproduction: Use /srv/storage/space/mirrors/gitorious.org/mnt/repositories/opensuse/kernel-source.git

repo = 'kernel-source.git'
origin_url = 'http://opensuse/kernel-source.git'

import logging
logging.basicConfig(level=logging.DEBUG)

from swh.loader.git.tasks import LoadDiskGitRepository

t = LoadDiskGitRepository()
t.run(origin_url=origin_url, directory=repo, date='2016-05-03T15:16:32+00:00')

output:

DEBUG:swh.scheduler.task.LoadDiskGitRepository:Creating git origin for http://opensuse/kernel-source.git
DEBUG:swh.scheduler.task.LoadDiskGitRepository:Done creating git origin for http://opensuse/kernel-source.git
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-scheduler/swh/scheduler/task.py", line 35, in run
    raise e from None
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-scheduler/swh/scheduler/task.py", line 32, in run
    result = self.run_task(*args, **kwargs)
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/tasks.py", line 39, in run_task
    return loader.load(origin_url, directory, dateutil.parser.parse(date))
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/base.py", line 422, in load
    self.fetch_data()
  File "/home/tony/work/inria/repo/swh/swh-environment/swh-loader-git/swh/loader/git/loader.py", line 48, in fetch_data
    type_name = self.repo[oid].type_name
  File "/usr/lib/python3/dist-packages/dulwich/repo.py", line 472, in __getitem__
    return self.object_store[name]
  File "/usr/lib/python3/dist-packages/dulwich/object_store.py", line 119, in __getitem__
    return ShaFile.from_raw_string(type_num, uncomp, sha=sha)
  File "/usr/lib/python3/dist-packages/dulwich/objects.py", line 393, in from_raw_string
    obj.set_raw_string(string, sha)
  File "/usr/lib/python3/dist-packages/dulwich/objects.py", line 305, in set_raw_string
    self.set_raw_chunks([text], sha)
  File "/usr/lib/python3/dist-packages/dulwich/objects.py", line 310, in set_raw_chunks
    self._deserialize(chunks)
  File "/usr/lib/python3/dist-packages/dulwich/objects.py", line 761, in _deserialize
    raise ObjectFormatException(e)
dulwich.errors.ObjectFormatException: invalid literal for int() with base 10: b'Thu Oct 30 17:30:06 2014'

Related T819

Revisions and Commits

rDLDG Git loader
rDLDGfece2335e246 swh.loader.git.loader: Warn when object malformed and continue

Related Objects
Search...

Event Timeline

ardumont renamed this task from Gitorious import: ObjectFormatException raised when badly formatted object to Gitorious import: ObjectFormatException raised when badly formatted object (around date?).Oct 27 2017, 10:59 AM
ardumont created this task.
ardumont added a parent task: T674: Gitorious import: Examine ingestion logs for errors and list them if any.
ardumont added a comment.EditedOct 27 2017, 11:52 AM

Debugging problematic object shows 1e82c9224b8898672b3b6fe8b6b737f7eed24cf6 which git fsck references as well.
Turns out it's a badly formatted tag:

error in tag 1e82c9224b8898672b3b6fe8b6b737f7eed24cf6: badDate: invalid author/committer line - bad date

Full output:

$ git fsck
Checking object directories: 100% (256/256), done.
error in tag 1e82c9224b8898672b3b6fe8b6b737f7eed24cf6: badDate: invalid author/committer line - bad date
...
Checking objects: 100% (342172/342172), done.
dangling tag 800cc01761a1c8afaff4c08dfa89ccc40a20b5b4
dangling tag 4d0db08e8248c026fb05147f15c0aa0e78a0e506
dangling tag 2f194020fe0ba8bd1c83ba5675f2410540a61f73
dangling tag e042809405ac13122ced7c2b7b0cda7e080c10bc
dangling tag 647e705587e0f2a6e644d7b4b3b10139f6977a12
dangling tag 0f8b302187cb4f560b1e595a82b7ac4206af5a62
dangling tag f192801f359c453048a96d29b2d7cdf1a1388f4b
dangling tag 44a47094246c92bbd371be9181da98da858df472
dangling tag d9e47061343c38833bf7c84aeff9991f3d1caf32
dangling tag 8ff870a3b720fdf00342be758f2fe51c5ef57c78
dangling tag 96fcf003a536f3eebe6f399257c5ed59f59e8730
dangling tag 15097138a80ae867aa9d0b9e0e3adf8489d5dc6a
dangling tag e921a1d5db103837e1ac2a7c233c833861542c1a
dangling tag 8930c1a41ef1d669855ac0c32f2f900766f657b8
dangling tag 1136f1c3ae2877536e2fee3b33df231a9641f59b
dangling tag ab37c162f236a1f2288ac6a765d5aca04f95d3f8
dangling tag ca4991a47fdfef373c4a97c463144113dc2ce42e
dangling tag 107fa114e16d35926728b6183be6c2283a0eb478
dangling tag 6890d14434ea58a61962380df3f613d66432cad4
dangling tag 7ad631a0b984f1b8d6a16155ad4b766c2363d8ee
dangling tag 0109829a8a48da7b6b40881e963172fbe1baf38c
dangling tag cc16829dd50ebff6acd1b9f4157264318ac29129
dangling tag 4649322b279f2787288fd92c884c176c55222c09
dangling tag de5cb2061e918de4d6c726b6920089aa3eac5068
dangling tag c585b2f1744894103db6a6d085c2ead74cbffddf
dangling tag cbafa23af8ad4139f2813f4f4250c5010cf6ca44
dangling tag 74c9d2f07c3b7c8cf39a353c60b79f1a7b0aea8b
dangling tag d527d37f7f8956bb40da3106b6e1a20e3ea7d375
dangling tag ec799343370adc42b9819229c16071abf61624be
dangling tag 3687e315c46a28426e9032fdb70e5dbf77bce1a4
dangling commit 449313b84bc37cd462f78861bd4c8caa4cfa5c1b
dangling tag d1aba37522abb924a212e92498c2f316b3037fa7
dangling tag 87b543eebfaeddf3db8a11c7ad38baea2ca60233
dangling tag 5ddff3ad567c65878d18d00abfa58c09a882aac5
dangling tag 80eba319a7c15bcdb45409b43414a267a196064c
dangling tag 78f4a3b314220572fb8893052307508b380990cd
dangling tag 00215495197fe3cd3e0608b73c55836ef9055b6f
dangling tag 1c420433cb3a3988d8597105238df556c75c9776
dangling tag ec600450ca34175733665b5cc85f794c8c549593
dangling tag 1275a45f3a55fa7656709defc586b3d1e54ac01d
dangling tag 4084542a2ac55cc7185deaf0551542a165a71268
dangling tag c094f4bc5b9fd6b3fd34af40d6ccc7e27f852414
dangling tag 709c24cd33f30293c07df0ab6010d1aa9dde72e8
dangling tag 8ca0648f8ee36b32212ad3e3bb59cfa3b8e9ac33
dangling tag 7ba194ed7abe73e90add2f806f8c306398b247f6
dangling tag 95c3043807d067d0fd6ed38d52f25e00a30407d8
dangling tag b3e9744b35845bb4ddc21db9d2cff1f31291b6df
dangling tag f5eaf41a7891608177f5740dbb2123ab5449e908
dangling tag 8ef9e4d4d7598098100882119b40e0370b551577
dangling tag e60925b0479d07bd5d7035c98bbdb18433b4d9dc
dangling tag 000f6546739da74d9f9fb7a29fb16f9f10d0ac24
dangling tag 1010c52ea9507f10fa8fcc654ab5313a116748e3
dangling commit b630d53dd44f4ae42900b21e02745c1c745728c8
dangling tag 4a78152a69087c1dcf4ae7c5e07292313d8579f3
dangling tag 7c9f55e5d2600661ac998907310673a48b0caf37
dangling tag 15ad25e0a6ccc281a96b25ba4fbe5b545b0dbdf5
dangling commit 53b125b3fd2a09215b1db47a67b07a1d82dabc15
dangling tag 364256365f7d7286697cdb76ed318706697a06df
dangling tag 845346cfd49fcdee8b3e33665eeab2d57f10f0ea
dangling tag ccbec6476571f4d8d24f37684519c054244e852d
dangling tag ebd73634f8672cd7e9adca1a7fb3a6f639b0ed0a
dangling commit bad9a65bccc6ef7ee5c42a68efaba3af01c05ce2
dangling tag ec05a7c7e89a3dab1577cd44a73c45bfa167cfcb
dangling tag 2e0b37b4b667ebb930a98329b6fe65a653804ea2
dangling commit c30b073fca66f60c8bc19a97c940235bc77888e6
dangling tag f32e372a19d844140a10241f2bc6a052a3863f59
dangling tag 4747b7c2193af11da1e66527f8c652acf6944719
dangling tag ee5237193ea80ac3b5c0ecea11a96b36b84f1fc7
dangling tag cd81b7a01fd7cc2decaadd2fbcad65f781e68e45
dangling tag 3689f72f5ce8ebb5d017f9feb0883df289af635a
dangling tag b3a187876e4da0d9d573e2728af3c733c0a38dc5
dangling tag 07b7b74aeb5c2836b59564fa0d865f72dabd0dbb
dangling tag 78c6376f6978d988d667e1d07af8661c47819b39
dangling tag f9cfb7f1c7d83574e22ec1128fa8fffa8811ceba
dangling tag e8e5877ad931e1024c8b1a2c4548411685821ef3
dangling tag ed04783c65b0c139aa701bec61cc910e6f51170b
dangling tag f70de8a912019114a377e1cf78a40d291be9cd57
dangling tag d734d879de1c962b3e4e64a69bcc4a44b83ff7b4
dangling tag 454a48f2abfed471f4f6bda274261266075f499c
dangling tag 5b4e48222a107f729f958de8acc868a8897c5560
dangling tag 116498bc82854880a3104de0782fb624b14d3851
dangling tag 5f72b8b0dc84ed09ab48ef5465b6b22d4d711c99
dangling tag 6e955841d5b9081b27d72a2b29379ce64091d6e5
dangling tag 8cb298eca3dede8d73879582e3ae0239c05718f9
dangling tag 02b8e8d3d6e03e4ef630def0078481e54139e136
dangling tag 87c86840b484f453fc0e109feb08cb72614f1f71
dangling tag 9ced68c287812b356e94c43d38bba9268e5cfd6e
dangling tag 74f508ae1ced06297bb35fcba08857da0a6920f9
dangling tag 2f0229c8abcfe632e56cec02d44fecadfbfbe0da
dangling tag e90e299ee737297f218857e101b3af194b1765f9
dangling tag b51a29488ee6a66b2b24fe6046e6e0f603a99953
dangling tag a61d29726a783d63ea92cf2223e88b7db05e65ea
dangling tag 582939ce5b076a1105e1ef07d090b1fb80236e68
dangling tag da33c90d9a8060cda9a7f31f0d92ad7b2622837e
dangling tag 04439942c5a956a52b2b2b3455f1845e0e6ca2fc
dangling tag 027f796bb7bc394035464b44d498da706a513efc
dangling tag 1e82c9224b8898672b3b6fe8b6b737f7eed24cf6
dangling tag 9fa67970dbee5177c40743f4e1673f7da7b0e3c2
dangling tag 17a7292d71d5d91632aa75b9ff6008b982f4500a
dangling tag 83c77937c47caddc2bfb7af228b25108e9e63765
dangling tag efcab9a5543e9967207f538b6c331d559be72518
dangling tag 23e60916eb04bacfee9ae1759e33d92168c579c8
dangling tag d3e9a98951d83b42f6275beb00276d60b511e9e0
dangling tag 69f6896bf6adea91cdac2feab96d03d59cee4342
dangling tag 75fb3978058a7d200b2c9042d708d0e6af021aec
dangling tag 3715aad5e46cf23be5d75ea0481af59246d763a9
dangling tag f0332a6a7c2dc53f6b0b4375e07e63f0db0bbbe6
dangling tag 3a582a3e6085d85f9878902a260a3e66541d1db0
dangling tag 045dba30d696a1fd0a106eb2bdcbc8e55472e6ee
dangling tag aa64fad5b8364dd4d68eb0132351b9ec72add9d6
dangling tag 4d875a9576691a8d5e31c074bfb3c159738677db
dangling tag 788dba250e620607647a9775acee48293bd7d392
dangling tag d88f0a2243d75f9725453b7b1056daeaeaa5b4a1
dangling tag 529c9a20d984c9d98e4546fc1f640e46f39a4006
dangling tag 712c0b01218107f272cfe61c49cf2097bc29b58f
dangling tag fa37dbf3e3f8f94768736fa85be45ebece9a2748
dangling tag 2e45dbb73e91b23ca72fa801b7a59a9a11dab508
dangling tag e348dbab5d933b88cd15cf61e432cf2f8427ec1e
dangling tag 6b5fabbabc34f8a17b74a85926c2254e04e44fef
dangling tag 8d659b143adf0cc1c563e1a9cffa768260478b0d
dangling tag 17832be5b9d6943c65a88c5b906c6469f6612004
dangling tag 4b9eabc49c0204c8eeba026634a0885d46f3dcd9
dangling tag 24ab1bc79518f16cd4c1f85745b4b33c25adf205
dangling tag 09ce4b79f504dd8a9868fd17f63b16bddef36b7d
dangling tag cc0d1c8e70f32fa2103f79ed7f26c3481e4fba6c
dangling tag 1641fc675b056e0e324d579bbabb708631479b73
dangling tag 205b0c5163c84fe45718630145768ca16f02ba83
dangling tag 2e7d9c255fdbb8a547919fd4cf5661f743133fdb
dangling tag 9c9ddc87b05b77013f36f2dda22773d1d0dc0227
dangling tag 4ca02c8c084c51e682425fa35039882ab106e27f
dangling tag 41ab6c353e8a7c3960156563a2f468779f9ed5cf
dangling commit 86bfec4ec3d52ef09abeb1fd8b74e7b58bd1d0e0
dangling commit c0c4cc06cb87ad787dc41c6c2b20e45aa724a26a
dangling tag 38136dcf8055f8902487bab906dc9eeedc993d47
dangling tag 7a3edd496ac280980e979a88c699a64122b1701d
dangling commit 96495d367ea0545053b7c677fade249bcb90abb9
dangling tag a158cd5043a6f3ad07979bf1bba21cb9ca717c95
dangling tag 5f82fdbed203dfac1aafae279803158b4dc5a56c
dangling tag 3acd2ddad81aed80199ffe039b04b6e28ee87e6a
dangling tag 9bde2d230fd6c828b0f5e8c0a334199ceff73a4a
dangling tag 2e001e114e9142177c89bdfd6558792a776d8775
dangling tag 05250e547d1f33517d25c187aa7b5e22d4164e78
dangling tag 28295e60e072595bc9f6777192348730a404cdf2
dangling tag c52daed58952c7b1c9b72f060c3a2bf22958e415
dangling tag 2a42de776d1531445dafea6dfa597bc76697f9e5
dangling tag 4646de0caf630e057fed16cb731370244c9d36f3
dangling tag 6154de6f190ce931c94c9e417a82501684f6a980
dangling tag a05d1e5e31ebf212372500d7c85872516a236fe6
dangling tag b85f4e9fcd86b865e5b9d70b3e8552b18f7b110d
dangling tag 39750e14c8a1742db8c10116be7daf664ff9f763
dangling tag 627c1effa14466c55dfe1e2d99f5a6e153e63300
dangling tag 068dce36d620a44333a9ce0203ae524a6b7cc561
dangling tag ad92defda44892fdb38da9a238b43a7fc3890e2c
dangling tag 36a49e0084df749608e36d92c7e141c850255e24
dangling tag 19b73e7717b0eaeadb3c90485156ee7323aa6e0b
dangling tag 7fcb9e5c2d75d6e86edbdad53edd5f3988562538
dangling tag 57d05e35f571be9a479310889c609443565aec0f
dangling tag 66e2ae696a46d9d2f2e3dd77cdcfb04f4af1b5f7
dangling tag 49fb7ecfee6d6c495bdd7da2e6b6b78b243a7179
dangling tag eb191f00cfdbce615df04efc98604de81a6020e8
dangling tag 00279f925f7408199a69c9638d4de08c1cc370a1
dangling tag 29281fd08c6a940cd06633bc598af272d8ea43ee
dangling tag 342a8f8026c0d19db9991352379f92f7654b43f3
dangling tag 4239cf6de11382d336b0fa29481e99c593d344af
dangling tag 22442f724c082628059ba38e4d8c0051ea9149f7
dangling tag 3081efda8df107439ba01750f7977238a68aee04
dangling tag ec900f89e1a5a93e91aa54651b94a390911f0085
dangling tag 11985ffbd6bdc66e715027ecd503f26ba19f301f
dangling tag cca44ff5b82bff843282a3b6468d7effc1b4f8ef
dangling tag 90b46fa540305ba34299aa4c4419eda531e7c9dd
dangling tag 93b5df91b224ffcddae7da883dcec3105c797b77
dangling tag b0c80f78f5c1082d816d03f53bfd8e273d6d5017
dangling tag cfd51f16b1aa0569f470dadae3b9b6440e9d3f90
ardumont updated the task description. (Show Details)Oct 27 2017, 12:25 PM
ardumont closed this task as Resolved by committing rDLDGfece2335e246: swh.loader.git.loader: Warn when object malformed and continue.Nov 7 2017, 6:22 PM
ardumont added a commit: rDLDGfece2335e246: swh.loader.git.loader: Warn when object malformed and continue.
gitlab-migration changed the task status from Resolved to Migrated.Jan 8 2023, 4:23 PM
gitlab-migration claimed this task.
gitlab-migration added a subscriber: gitlab-migration.

This task has been migrated to GitLab.

About · Developer information · Legal