I am very new to Spark.
datafrom kafka is consumed in Spark
datanext needs to be processed by querying database to get the relevant formula for manipulation
- The thus
processed datais then used in getting RDD – mapaggregatereduce… to make the
processed dataare then stored in DB.
Is it right to do this in Spark? i.e can we query database to get a formula to process the data consumed in Spark
If not, please guide me on how I need to be achieving this.