[BUG] Remote Finder unable to parse UTF-8 characters

[jbradleynh]

Describe the bug
When using the remote finder to look for metrics from multiple backends, any non ascii character results in an exception when trying to unpickle the response.

To Reproduce
Steps to reproduce the behavior:

• create a metric with with a non-ascii character in its name or path

echo "test.random.død 123 date +%s" |nc ${carbon-host} 2003

• attempt to expand metrics finder in graphite web

Expected behavior
Be able to see all metrics present, including the ones with UTF-8 characters in their namespace

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS flavor: Ubuntu (Xenial)
• Graphite-web version: 1.1.7, master
• Django/Python version: 2.2.16/3.5.2
• Setup type: from sources

Additional context

This was tested on originally on v1.1.7, as well as on the master branch with the environment set for UTF8_METRICS=true, related to #2641

Trying to expand the metric tree in the UI or trying to render this metric results in the following error

2020-11-23,22:11:43.356 :: RemoteFinder[carbon-1:8080] Error decoding response from http://carbon-1:8080/render/?format=pickle&local=1&noCache=1&from=1606168063&until=1606169503&target=test.random.%2A&now=1606169503: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)
Traceback (most recent call last):
  File "/opt/graphite/webapp/graphite/finders/remote.py", line 291, in deserialize
    return self._deserialize_stream(reader, result.getheader('content-type'))
  File "/opt/graphite/webapp/graphite/finders/remote.py", line 319, in _deserialize_stream
    data = unpickle.load(stream)
  File "/opt/graphite/webapp/graphite/util.py", line 219, in load
    return SafeUnpickler(file).load()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 13: ordinal not in range(128)

It may be the case that the some means to specify encoding='bytes' needs to be passed to:

https://github.com/graphite-project/graphite-web/blob/master/webapp/graphite/util.py#L194

[piotr1212]

Yes, I expect changing to .load(encodigin='bytes') should fix this. Any chance you could test this?

[jbradleynh]

@piotr1212 I just tried out the branch you opened however, I'm seeing only the following from the info.log with no results being returned:

root@5aafcd5e0e5a:/var/log/graphite# cat info.log
2020-11-24,14:18:32.355 :: Default user didn't exist, created it
2020-11-24,14:18:32.372 :: Default profile didn't exist, created it
2020-11-24,14:20:12.566 :: graphite.finders.remote.find_nodes :: failed in 0.018878s
2020-11-24,14:20:12.567 :: Exception during find <FindQuery: * from * until *> after 0.026654s: 'NoneType' object has no attribute 'split'
2020-11-24,14:20:12.568 :: graphite.finders.remote.find_nodes :: failed in 0.0217464s
2020-11-24,14:20:12.568 :: Exception during find <FindQuery: * from * until *> after 0.028288s: 'NoneType' object has no attribute 'split'
2020-11-24,14:20:12.569 :: find_view query=* local_only=0 matches=0
2020-11-24,14:20:12.569 :: received remote find request: pattern=* from=None until=None local_only=0 format=treejson matches=0

[piotr1212]

What is your remote finder? Another Graphite-web? Have any exceptions there? Do you see regressions with this branch (ie metrics with only ascii still work)?

[jbradleynh]

The remote finders are go-carbon carbonservers, and returning data in a pickle format. I'm able to see that it is responding to the find requests as expected from a tcpdump, without any error. As a test all data was purges and only sample test data which uses ascii only characters is being generated, however with this patch in place unfortunately no data is being able to be parsed from the carbonservers by graphite-web

[adamcstephens]

I confirmed what @jbradleynh was seeing, that the patch is a regression and produces no results for ascii only metrics.

Switching to encoding="utf8" works and solves this issue while still supporting ascii only metrics.