filesystems - spark ssc.textFileStream is not streamining any files from directory -


I am trying to execute under code with 2 employees (meowe conferene) under code and 2 in each There are cores or efforts have been made with it also submit spark. Public class StreamingWorkCount Serializable {public static void (String [] args) {logger.getLogger ("org.apache.spark"). SetLevel (Level.WARN); JavaStreaming Contact jssc = New Java Streaming Contains ("Spark: 1952.168.19 9: 7077", "Java wordcount", new period (1000)); JavaDStream & LT; String & gt; TrainingData = jssc.textFileStream ("/ home / bdi-user / skill-drive / spark / data / training"). Cash (); TrainingData.foreach (new function & lt; JavaRDD & lt; string & gt;; zero & gt; () {public zero call (JavaRDD & lt; string & gt; string & gt; production = rdd.collect (); RDD) exception {println ("Received from sentence files" +); Returns from the {lt; return tap;}}); TrainingData.print (); Jssc.start (); Jssc.awaitTermination (); }}

and the log of that code

  15/01/22 21:57:13 INFO FileInputDStream: New files on time 1421944033000 MS: 15 / 22/01 21:57:13 Information JobScheduler: Added jobs for the time 1421944033000 MS 15/01/22 21:57:13 Information JobScheduler: 1421944033000 ms.0 from the working time of work start work 1421944033000 MS 15/01 / 22 21:57:13 Information SparkContext: Starting work: StreamingKMean.java:33 15/01/22 21:57:13 Information: foreach on DAGScheduler: Job 3 End: foreach on StreamingKMean.java33, 0.0000 9 4 sentences Files collected from [] ------------------------------------------- 15/01/22 21:57:13 Junk JobScheduler: Job's Job Streaming Job 1421944033000 ms.0 From Time to Job Set 1421944033000 MS Time: 1421944033000 MS ----------------------- ---- ---------------- 15/01/22 21:57:13 Information JobScheduler: Job Streaming Job Started 1421944033000 ms.1 Time of Job 1421944033000 MS 15 / 01/22 21 : 57: 13 Information JobScheduler: Job Streaming Job Job 1421944033000 MS .1 Time 1421944033000 MS 15/01/22 21:57:13 Information JobScheduler Set: Total Delay: 0.028 for Time 1421944033000 MS (Execution: 0.013) 15/01/22 21:57:13 Info MappedRDD: RDD 5 Nick Releasing List 15/01/22 21:57:13 Information Blocker: RDD Removal 5 15/01/22 21:57:13 INFO FileInputDStream: 1421943973000 Approved older files older than MS: 15/01/22 21:57: 13 Info FileInputDStream: Cleared 0 old files that were over 1421943973000 MS: 15/01/22 21:57:13 Information ReceivedBlockTracker: Removing batches ArrayBuffer ()  

The problem is that, I can not find the file form which is in the directory Please help me.

Try it with another directory and then copy these files to that directory while the job is running is.


Comments