Changeset View
Changeset View
Standalone View
Standalone View
swh/storage/postgresql/db.py
| Show First 20 Lines • Show All 965 Lines • ▼ Show 20 Lines | class Db(BaseDb): | ||||
| def stat_counters(self, cur=None): | def stat_counters(self, cur=None): | ||||
| cur = self._cursor(cur) | cur = self._cursor(cur) | ||||
| cur.execute("SELECT * FROM swh_stat_counters()") | cur.execute("SELECT * FROM swh_stat_counters()") | ||||
| yield from cur | yield from cur | ||||
| def origin_add(self, url, cur=None): | def origin_add(self, url, cur=None): | ||||
| """Insert a new origin and return the new identifier.""" | """Insert a new origin and return the new identifier.""" | ||||
| insert = """INSERT INTO origin (url) values (%s) | insert = """INSERT INTO origin (url) values (%s) | ||||
| RETURNING url""" | ON CONFLICT DO NOTHING | ||||
| """ | |||||
| cur.execute(insert, (url,)) | cur.execute(insert, (url,)) | ||||
| return cur.fetchone()[0] | return cur.rowcount | ||||
ardumont: ok for the main change but that one makes me wonder.
If the returned result is uneeded [1]… | |||||
Done Inline ActionsThe return value is used in postgresql.Storage.origin_add to return a counter of added origins: [...]
added = 0
for url in to_add:
if db.origin_add(url, cur):
added += 1
return {"origin:add": added}douardda: The return value is used in postgresql.Storage.origin_add to return a counter of added origins… | |||||
Not Done Inline Actionsright, i had forgotten that part. Thanks. So i gather cur.rowcount somehow returns something falsy [1] when the on conflict [1] this looks fine: In [11]: 0 if None else 1 Out[11]: 1 In [12]: 0 if False else 1 Out[12]: 1 In [13]: 0 if 0 else 1 Out[13]: 1 ardumont: right, i had forgotten that part.
Thanks.
So i gather `cur.rowcount` somehow returns… | |||||
Done Inline ActionsI was unsure of the behavior of cur.rowcount with the 'on conflict', so I checked (pg13 in a pifpaf session) and it seems to work as expected (aka only count the number of inserted rows): In [22]: c.execute("select * from origin")
In [23]: c.fetchall()
Out[23]: [(1, 'http://foo'), (2, 'http://bar'), (8, 'http://baz'), (11, 'http://biz')]
In [31]: execute_values(c, "INSERT INTO origin (url) values %s ON CONFLICT DO NOTHING", (("http://bar",),("http://toto",), ("http://foo",), ("http://tutu",) ))
In [32]: c.rowcount
Out[32]: 2douardda: I was unsure of the behavior of cur.rowcount with the 'on conflict', so I checked (pg13 in a… | |||||
| origin_cols = ["url"] | origin_cols = ["url"] | ||||
| def origin_get_by_url(self, origins, cur=None): | def origin_get_by_url(self, origins, cur=None): | ||||
| """Retrieve origin `(type, url)` from urls if found.""" | """Retrieve origin `(type, url)` from urls if found.""" | ||||
| cur = self._cursor(cur) | cur = self._cursor(cur) | ||||
| query = """SELECT %s FROM (VALUES %%s) as t(url) | query = """SELECT %s FROM (VALUES %%s) as t(url) | ||||
| ▲ Show 20 Lines • Show All 456 Lines • Show Last 20 Lines | |||||
ok for the main change but that one makes me wonder.
If the returned result is uneeded [1], might as well change the signature to None.
Also, please add type to the impacted method.
[1] that's somehow implicit here as the meaning in the returned value is not the same.