diff options
author | Peter Lamut <plamut@users.noreply.github.com> | 2020-06-09 20:48:10 +0200 |
---|---|---|
committer | GitHub <noreply@github.com> | 2020-06-09 18:48:10 +0000 |
commit | 74e0b0f8387207933c120af15b2bb5d175dd8f84 (patch) | |
tree | e8fe0a71fbfa6d2f59dece58a6b210f620111084 /tests | |
parent | 945bafc47cfaf79f0947c970dba17dc8fbe3e197 (diff) | |
download | python-api-core-74e0b0f8387207933c120af15b2bb5d175dd8f84.tar.gz |
feat: allow disabling response stream pre-fetch (#30)
Closes #25.
This PR adds the ability to disable automatically pre-fetching the first item of a stream returned by `*-Stream` gRPC callables. This hook will be used in PubSub to fix the [stalled stream issue](https://github.com/googleapis/python-pubsub/issues/93), while also not affecting Firestore, since the default behavior is preserved.
I realize the fix is far from ideal, but it's the least ugly among the approaches I tried, e.g. somehow passing the flag through `ResumableBidiRpc` (it's a messy rabbit hole). On the PubSub side monkeypatching the generated SubscriberClient will be needed, but it's a (relatively) clean one-liner:
```patch
diff --git google/cloud/pubsub_v1/gapic/subscriber_client.py google/cloud/pubsub_v1/gapic/subscriber_client.py
index e98a686..1d6c058 100644
--- google/cloud/pubsub_v1/gapic/subscriber_client.py
+++ google/cloud/pubsub_v1/gapic/subscriber_client.py
@@ -1169,6 +1169,8 @@ class SubscriberClient(object):
default_timeout=self._method_configs["StreamingPull"].timeout,
client_info=self._client_info,
)
+ # TODO: explain this monkeypatch!
+ self.transport.streaming_pull._prefetch_first_result_ = False
return self._inner_api_calls["streaming_pull"](
requests, retry=retry, timeout=timeout, metadata=metadata
```
If/when we merge this, we should also release it, and then we can add `!= 1.17.0` to the `google-api-core` version pin in PubSub.
### PR checklist
- [x] Make sure to open an issue as a [bug/issue](https://github.com/googleapis/python-api-core/issues/new/choose) before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
- [x] Ensure the tests and linter pass
- [x] Code coverage does not decrease (if any source code was changed)
- [x] Appropriate docs were updated (if necessary)
Diffstat (limited to 'tests')
-rw-r--r-- | tests/unit/test_grpc_helpers.py | 13 |
1 files changed, 13 insertions, 0 deletions
diff --git a/tests/unit/test_grpc_helpers.py b/tests/unit/test_grpc_helpers.py index 1fec64f..ef84514 100644 --- a/tests/unit/test_grpc_helpers.py +++ b/tests/unit/test_grpc_helpers.py @@ -80,6 +80,19 @@ def test_wrap_stream_okay(): assert responses == expected_responses +def test_wrap_stream_prefetch_disabled(): + responses = [1, 2, 3] + iter_responses = iter(responses) + callable_ = mock.Mock(spec=["__call__"], return_value=iter_responses) + callable_._prefetch_first_result_ = False + + wrapped_callable = grpc_helpers._wrap_stream_errors(callable_) + wrapped_callable(1, 2, three="four") + + assert list(iter_responses) == responses # no items should have been pre-fetched + callable_.assert_called_once_with(1, 2, three="four") + + def test_wrap_stream_iterable_iterface(): response_iter = mock.create_autospec(grpc.Call, instance=True) callable_ = mock.Mock(spec=["__call__"], return_value=response_iter) |