You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently both bigtable::AdminClient and bigtable::Client use grpc::GoogleDefaultCredentials, or grpc::InsecureChannelCredentials if taking to the Cloud Bigtable emulator.
We need to support any sort of credentials, this means that the credentials should be a parameter, or set via a policy. Most likely the latter, because some credential types (notably oauth credentials) require a periodic refresh (see GoogleRefreshTokenCredentials in the docs)
One of the problems is to refresh the credentials automatically, not hard if we have an async client, or if we are willing to keep an extra thread to perform the task.
@sduskis I will use this bug to track the issue with supporting other credential types, including the ones that need refreshing. You should let us know if this should be part of the MVP or not.
This comment is just to record what I have discovered so far. It may seem obvious to folks with more experience with web authentication, but some of us do not know anything about web authentication (e.g., me!), so it might be useful.
Connecting to the Cloud Bigtable Emulator
If you are running against the Cloud Bigtable Emulator you really do not need (or want) authentication, so we need to use grpc::InsecureChannelCredentials.
The bigtable::*Client classes already take care of that.
Connecting to Cloud Bigtable with a Service Account
Here, you may want to read the general intro to authentication in gRPC. My interpretation of this document is that basically grpc::GoogleDefaultCredentials works great if you are using a service account to access Cloud Bigtable. There are two sub-cases:
If you are running in GCE or GAE, the VM will have all the configuration you need to run with that service account.
If you are running outside GCE or GAE, you need to download the private keys of that service account to a file, and set the GOOGLE_APPLICATION_CREDENTIALS to the path of that file
Connecting to Cloud Bigtable with an OAuth 2.0 Token
That leaves basically one other case: you are running an application that cannot use a service account, for example, some kind of proxy that will access Cloud Bigtable on behalf of multiple users. I really cannot tell how common this is, but seems like something we need to support.
In this case you may want to start with the general documentation about using OAuth 2.0 with googleapis.
I expect this tool to be useful in this case. It can create an access (and refresh?) token for you. It needs the scopes for Cloud Bigtable, which can be found here:
The workflow is to obtain a refresh token using the browser (who is supposed to ask the client "do you want to use to access Cloud Bigtable). The application uses the refresh token to get access tokens, the refresh tokens are supposed to be kept secret but last longer than access tokens. The access tokens is what we need to pass to grpc::AccessTokenCredentials. The refresh tokens are what we pass to grpc::RefreshTokenCredentials.
Connecting to Cloud Bigtable with SSL Authentication
This is not supported by Google Cloud Platform. The only supported authentication in GCP are user accounts (via OAuth2), service accounts, and API Keys. API Keys are not applicable for Cloud Bigtable.
Connecting to Cloud Bigtable with Cronet
TODO - fill out this section. Not clear how this is used, maybe from a Chromium or Android project?
cc: @sduskis I had completely forgotten about this one. I believe this should just work right out of the box, we just use the gRPC classes for Credentials. But somebody should check, maybe write a couple of tests.
Use gRPC Credential Classes with GCP Bigtable c++ client
Before you begin
Set the Environment Variables
Following environment variables must be set to connect to InstanceAdmin or TableAdmin endlpoints
GOOGLE_APPLICATION_CREDENTIALS export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
where [PATH] is file path for JSON file that contains either user account or Service account key. Please refer this link for more details on Google Authentication.
GRPC_DEFAULT_SSL_ROOTS_FILE_PATH export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH="[PATH]"
where [PATH] is file path for gRPC Root Certificate. Please refer this link for more details on gRPC Root Certifacte Configuration.
Use Access Token Credentials
Acquire Access Token
Use gcloud to acquire an Access Token
Sample code to connect to InstanceAdmin endpoint
#include "google/cloud/bigtable/instance_admin.h"
#include "google/cloud/bigtable/instance_admin_client.h"
int main(int argc, char* argv[]) try {
std::string const project_id = argv[1];
// Put Acquired Access Token
grpc::string access_token("Valid Access Token");
auto call_credentials = grpc::AccessTokenCredentials(access_token);
auto channel_credentials = grpc::SslCredentials(grpc::SslCredentialsOptions());
auto credentials = grpc::CompositeChannelCredentials(channel_credentials, call_credentials);
auto instance_admin_client(
google::cloud::bigtable::CreateDefaultInstanceAdminClient(
project_id, google::cloud::bigtable::ClientOptions(credentials)));
google::cloud::bigtable::InstanceAdmin instance_admin(instance_admin_client);
auto instances = instance_admin.ListInstances();
return 0;
} catch (std::exception const& ex) {
std::cerr << "Standard C++ exception raised: " << ex.what() << std::endl;
return 1;
}
Use Refresh Token Credentials
Sample code to connect to InstanceAdmin endpoint
#include "google/cloud/bigtable/instance_admin.h"
#include "google/cloud/bigtable/instance_admin_client.h"
int main(int argc, char* argv[]) try {
std::string const project_id = argv[1];
grpc::string refresh_token = R"""({
"client_id": "XXXXXX.apps.googleusercontent.com",
"client_secret": "<actual secret here>",
"refresh_token": "<actual token value here>",
"type": "authorized_user"
})""";
auto call_credentials = grpc::GoogleRefreshTokenCredentials(refresh_token);
auto channel_credentials = grpc::SslCredentials(grpc::SslCredentialsOptions());
auto credentials = grpc::CompositeChannelCredentials(channel_credentials, call_credentials);
auto instance_admin_client(
google::cloud::bigtable::CreateDefaultInstanceAdminClient(
project_id, google::cloud::bigtable::ClientOptions(credentials)));
google::cloud::bigtable::InstanceAdmin instance_admin(instance_admin_client);
auto instances = instance_admin.ListInstances();
return 0;
} catch (std::exception const& ex) {
std::cerr << "Standard C++ exception raised: " << ex.what() << std::endl;
return 1;
}
#include "google/cloud/bigtable/instance_admin.h"
#include "google/cloud/bigtable/instance_admin_client.h"
int main(int argc, char* argv[]) try {
std::string const project_id = argv[1];
auto call_credentials = grpc::GoogleComputeEngineCredentials();
auto channel_credentials = grpc::SslCredentials(grpc::SslCredentialsOptions());
auto credentials = grpc::CompositeChannelCredentials(channel_credentials, call_credentials);
auto instance_admin_client(
google::cloud::bigtable::CreateDefaultInstanceAdminClient(
project_id, google::cloud::bigtable::ClientOptions(credentials)));
google::cloud::bigtable::InstanceAdmin instance_admin(instance_admin_client);
auto instances = instance_admin.ListInstances();
return 0;
} catch (std::exception const& ex) {
std::cerr << "Standard C++ exception raised: " << ex.what() << std::endl;
return 1;
}
Note :- One may face "Request had insufficient authentication scopes." error while running above example. This might be due to disabled "Cloud API access scope" for Bigtable. This error can be removed by providing sufficient access as explained here.
@coryan
An attempt with IAM Credentials resulted into "Received RST_STREAM with error code 1 [13=INTERNAL] " from gRPC server. I have updated these credentials as "not supported".
Do we keep the bug open and change the milestone or create a new issue just for grpc::GoogleIAMCredentials()?
Activity
coryan commentedon Dec 22, 2017
@sduskis I will use this bug to track the issue with supporting other credential types, including the ones that need refreshing. You should let us know if this should be part of the MVP or not.
coryan commentedon Dec 25, 2017
This comment is just to record what I have discovered so far. It may seem obvious to folks with more experience with web authentication, but some of us do not know anything about web authentication (e.g., me!), so it might be useful.
Connecting to the Cloud Bigtable Emulator
If you are running against the Cloud Bigtable Emulator you really do not need (or want) authentication, so we need to use
grpc::InsecureChannelCredentials.The
bigtable::*Clientclasses already take care of that.Connecting to Cloud Bigtable with a Service Account
Here, you may want to read the general intro to authentication in gRPC. My interpretation of this document is that basically
grpc::GoogleDefaultCredentialsworks great if you are using a service account to access Cloud Bigtable. There are two sub-cases:GOOGLE_APPLICATION_CREDENTIALSto the path of that fileConnecting to Cloud Bigtable with an OAuth 2.0 Token
That leaves basically one other case: you are running an application that cannot use a service account, for example, some kind of proxy that will access Cloud Bigtable on behalf of multiple users. I really cannot tell how common this is, but seems like something we need to support.
In this case you may want to start with the general documentation about using OAuth 2.0 with googleapis.
I expect this tool to be useful in this case. It can create an access (and refresh?) token for you. It needs the scopes for Cloud Bigtable, which can be found here:
https://github.com/googleapis/googleapis/blob/master/google/bigtable/bigtable.yaml
https://github.com/googleapis/googleapis/blob/master/google/bigtable/admin/bigtableadmin.yaml
The workflow is to obtain a refresh token using the browser (who is supposed to ask the client "do you want to use to access Cloud Bigtable). The application uses the refresh token to get access tokens, the refresh tokens are supposed to be kept secret but last longer than access tokens. The access tokens is what we need to pass to
grpc::AccessTokenCredentials. The refresh tokens are what we pass togrpc::RefreshTokenCredentials.Connecting to Cloud Bigtable with SSL Authentication
This is not supported by Google Cloud Platform. The only supported authentication in GCP are user accounts (via OAuth2), service accounts, and API Keys. API Keys are not applicable for Cloud Bigtable.
Connecting to Cloud Bigtable with Cronet
TODO - fill out this section. Not clear how this is used, maybe from a Chromium or Android project?
https://chromium.googlesource.com/chromium/src/+/master/components/cronet?autodive=0%2F%2F
coryan commentedon Dec 28, 2017
Here is a relevant bit of info about how to refresh credentials:
https://github.com/GoogleCloudPlatform/cloud-bigtable-client/blob/master/bigtable-client-core-parent/bigtable-client-core/src/main/java/com/google/cloud/bigtable/grpc/io/RefreshingOAuth2CredentialsInterceptor.java
coryan commentedon Jun 14, 2018
cc: @sduskis I had completely forgotten about this one. I believe this should just work right out of the box, we just use the gRPC classes for Credentials. But somebody should check, maybe write a couple of tests.
2 remaining items
manish-qlogic commentedon Jul 19, 2018
Use gRPC Credential Classes with GCP Bigtable c++ client
Before you begin
Set the Environment Variables
Following environment variables must be set to connect to InstanceAdmin or TableAdmin endlpoints
GOOGLE_APPLICATION_CREDENTIALS
export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"where [PATH] is file path for JSON file that contains either user account or Service account key. Please refer this link for more details on Google Authentication.
GRPC_DEFAULT_SSL_ROOTS_FILE_PATH
export GRPC_DEFAULT_SSL_ROOTS_FILE_PATH="[PATH]"where [PATH] is file path for gRPC Root Certificate. Please refer this link for more details on gRPC Root Certifacte Configuration.
Use Access Token Credentials
Use gcloud to acquire an Access Token
Use Refresh Token Credentials
Use JWT Access Token Credentials
Use Google Compute Engine Credentials
Note :- One may face "Request had insufficient authentication scopes." error while running above example. This might be due to disabled "Cloud API access scope" for Bigtable. This error can be removed by providing sufficient access as explained here.
Use of IAM Credentials
These credentials are not supported.
manish-qlogic commentedon Jul 25, 2018
@coryan
An attempt with IAM Credentials resulted into "Received RST_STREAM with error code 1 [13=INTERNAL] " from gRPC server. I have updated these credentials as "not supported".
Do we keep the bug open and change the milestone or create a new issue just for
grpc::GoogleIAMCredentials()?coryan commentedon Jul 26, 2018
I just changed the milestone. Let's keep this bug open for the moment. I will unassigned it too since you have done all that can be done here.
[-]Support all grpc Credential classes.[/-][+]Test with all gRPC Credential classes.[/+]coryan commentedon May 2, 2019
After re-reading the bug, we are done here.
evanj commentedon Sep 18, 2019
This should be documented somewhere! Like maybe in the examples, or in the gRPC documentation? This is very hard to figure out on your own.
coryan commentedon Sep 19, 2019
@evanj thanks, we will lift the results from here to some examples.