nlp – ResourceExhausted: 429 Quota metric API quota for natural language exceeded dataflow with Python SDK

I'm creating a Dataflow pipeline to read CSV, do a sentiment analysis on the Google Cloud NLP API, and send the result to BigQuery.

When the function performing the sentiment analysis retrieves the collection, the error mentioned above is displayed to me.

I'm considering splitting the pcollection into a small pcollection to handle the offer restriction in the NLP API.

(p
       | 'ReadData' >> beam.io.textio.ReadFromText(src_path)
       | 'ParseCSV' >> beam.ParDo(Analysis())
       | 'WriteToBigQuery' >> ...
)