Paste P1532

.csv.zst to .csv.zst
ActivePublic
Actions

Authored by vlorentz on Nov 30 2022, 1:39 PM.

Tags

None

Subscribers

None

	# Open temporary output for writes as CSV
	with tmp_output_path.open("r") as output_fd:
	with ZstdDecompressor(level=19).stream_reader(input_fd) as zstd_write:
	with csv.writer(zstd_writer) as csv_writer:
	# write header
	csv_writer.write(
	("origin_swhid", "person_base64", "person_escaped")
	)

	# Open input for reads as CSV
	with self.origin_contributors_path.open("r") as input_fd:
	with ZstdDecompressor().stream_reader(input_fd) as zstd_reader:
	with csv.reader(zstd_reader) as csv_reader:
	for (origin_swhid, person_id) in csv_reader:
	(name, escaped_name) = person_id_to_names[person_id]
	base64_name = base64.b64encode(name)
	csv_writer.write(
	(origin_swhid, base64_name, escaped_name)
	)

Event Timeline

vlorentz created this paste.Nov 30 2022, 1:39 PM