Rebased to remove doc and parse_url commits

Fixed the cherry-picked commit

Re-submit

Inlined the intersting fix

I would add that I have even been bitten by it, providing a "http://" url that became "https://"http://""... :)

Hm I submitted 2 times this commit, but the last one is D4130.
Sorry I will close this. Config related code will go to core.config, yes, when it will be ready for it.

OTOH it is CLI so interactive use so the error is instant.
The function also had a bad name.
I can just keep adding the trailing slash if you prefer, inline (no specific function).

• Rename scan to do_scan in scanner
• Fix not self-contained temp_path fixture in conftest
• Change tests and fix behavior of scanner cli

Renamed commit

If my last comment was unclear:

• concrete storages having different default does not matter since they are the endpoints in this chain.
• any other non-terminal impl the buffer interface, if there may exist, must follow some rules: pass arguments unprocessed (*args) or a subset of BPS.OBJECT_TYPES.

OK I have been biten by the tests I just introduced. They are not self-contained, advice welcome on how to make them so.

If whatever component before BPS has its flush called with no object_types, it depends on the impl of this component how it chains well. As is, it should either chain down the arg unprocessed (from *args/**kwargs for example, what I have done for all proxies before I understood the getattr hooks), call it without this arg or give it a valid value like OBJECT_TYPES (which could be shared). In short intermediate should not have an invalid default passed down verbatim.
The case of concrete storage is not clear, because they don't chain call it nor need it in the first place. So any value is OK.
We have no other component than BPS that would do something with this (buffering) interface though, and it maybe would not make sense to have another.

Tested on storage and loader-core as seen with mr grep '\.flush('.

resubmit after rebase

Superseded by D4033 D4034

Can you show me the code you tried?

commit has changed hash

commit has changed hash

IMO it's more readable like this

Also, I added description of defaults in the command help, but it reads badly, do you know a better way?
Remarks:

• clicks inline even if I use a docstring
• moving it to the command/group docstring would resolve the problem, but causes unexpectedly the docstring to be ignored by click (we wants static strings here)

extracted isort change from commit, which was directly merged by olasd

In D4046#100086, @vlorentz wrote:

fail if file does not exist instead of using empty config dict

Isn't it already the default behavior?

on another note, do I write tests for this cli?

Last attempt:
https://docs.python.org/3/reference/datamodel.html#objects-values-and-types
clearly states what is object identity and that it does not change and that we compare it with is.

Wanted also to rename constructor parameters but they are tied to the config parameters.

@zack @olasd @vlorentz I am puzzled. Do I go with the ImmutableDict solution or throw away another part of what I did the last 3 days (did it also in another commit)? Already removed code that was against the strict signature checker in the unit tests.

@vlorentz: true, this is bad. I added the check after, and didn't check this...

And look! I can get rid of the identity check, just it would merge with itself. This one is to avoid perf penalty.

The fact that is not so common a pattern does not make it bad.
It IS exactly like using None if we make this dict immutable, with added benefits of readability and checking. It even has no performance penalty or anything.

In D4017#99688, @olasd wrote:

I see that e37f63930b323a1e811861d298e62516ed94649a has been pushed. Meh.

To properly push after a rebase, you need to do arc diff --update to make sure that the changes after rebase will be picked up by phabricator and the diff will be automatically closed.

I view it that way:

• identity checking a default value is always good, even if mutable, precisely because static values like these can never change identity (identity/value dichotomy). It would be wrong to value equality check it through if it is mutable of course;
• this value can only be modified inside the module (modulo import time hacking), so the question are:
  • • do we want it immutable (we don't plan to have code that mutate it)?
    • trust the module code for well behaving? it is a real policy question, I think your answer is no, but then why manual replacing None if we can do without it?
  • this pattern is equivalent to the None one, except we statically (as opposed to manually replace None by a default) never allow any invalid type like NoneType. I try to leverage static typing to the fullest, and avoid the potential "billion dollar mistake".

After this rationale, which could be a valid policy to introduce in the project, now the proposition is:

repush?

repush?

Found how to do write passtrough functions while having good default handling and keep the signature checker happy. A bit hacky.

Fix typo and remove TODO comment

build failed and one thing to rework

I think you should not remove this code as it has one use case (filename is provided as parameter, from CLI or not) but have a different function name for the "prod" use case (e.g. parse_config_file_envvar).
Some parts in the old code are not needed anymore, but not all. In another diff we will cover the other use case.
For now better to have 2 functions.

dedented debian packages and updated one description