Data transfer between Async boundaries can be implemented in following ways namely:
Pull Model: Scenario where Consumer requests data from Producer. It works well when the Producer is faster than Consumer. Push Model: Scenario where Producer pushes data to Consumer. It works well when the Consumer is faster than Producer. Dynamic Push/Pull Model: In this model Consumer Signals Producer that it is ready to accept N elements and Producer then sends Consumer N elements when they are ready.
0 Comments
Data set:Online news popularity
I will be using Casbah Scala library to write the data from csv file into mongoDB. And then write some mongo queries to understand the data. MongoConnection.scalapackage com.examples.onlineNewsPopularity import com.mongodb.casbah.Imports._ object MongoConnection { val mongoClient = MongoClient("localhost", 27017) val mongoDB = mongoClient("test") } Main.scalapackage com.examples.onlineNewsPopularity import com.mongodb.casbah.Imports._ import Constants._ import scala.io.Source import MongoConnection._ object Main { def main (args: Array[String]): Unit = { val COLLECTION_NAME = "OnlineNews" if(!mongoDB.collectionExists(COLLECTION_NAME)) { mongoDB.createCollection(COLLECTION_NAME, DBObject.empty) } val lines = Source.fromFile("OnlineNewsPopularity/OnlineNewsPopularity.csv").getLines().drop(1).toSeq println(lines.length) lines foreach { line => val components = line.split(",") val eachObject: DBObject = { val stringComponents = (for { i <- 0 to 2 } yield { DBObject(headers(i) -> components(i)) }) reduceOption { _ ++ _ } getOrElse DBObject.empty val intComponents = (for { i <- 3 to 16 } yield { DBObject(headers(i) -> components(i).toInt) }) reduceOption { _ ++ _ } getOrElse DBObject.empty val doubleComponents = (for { i <- 17 to 33 } yield { DBObject(headers(i) -> components(i).toDouble) }) reduceOption { _ ++ _ } getOrElse DBObject.empty stringComponents ++ intComponents ++ doubleComponents } mongoDB.getCollection(COLLECTION_NAME).insert(eachObject) } } } Query to find shares distribution across weekdays
|
ArchivesCategories |