csv.writer examines every character of every field for membership in
dialect.lineterminator. In join_append_data() (Modules/_csv.c) that test is
a call to PyUnicode_FindChar(), made once per character per pass — and the
function runs two passes over each field, one to size the record buffer and one
to fill it.
perf record on 2000 rows x 4 fields attributes 54% of cycles to
PyUnicode_FindChar:
54.12% python PyUnicode_FindChar
2.02% python PyUnicode_FindChar@plt
25.57% _csv..so join_append
3.12% python listiter_next
Because a dialect's terminator is immutable, we can compute its highest code point once at construction and use that information to avoid unnecessary work.
(PR is already prepped)
Linked PRs
csv.writerexamines every character of every field for membership indialect.lineterminator. Injoin_append_data()(Modules/_csv.c) that test isa call to
PyUnicode_FindChar(), made once per character per pass — and thefunction runs two passes over each field, one to size the record buffer and one
to fill it.
perf recordon 2000 rows x 4 fields attributes 54% of cycles toPyUnicode_FindChar:Because a dialect's terminator is immutable, we can compute its highest code point once at construction and use that information to avoid unnecessary work.
(PR is already prepped)
Linked PRs