filesystems - spark ssc.textFileStream is not streamining any files from directory -
I am trying to execute under code with 2 employees (meowe conferene) under code and 2 in each There are cores or efforts have been made with it also submit spark. and the log of that code The problem is that, I can not find the file form which is in the directory Please help me.
15/01/22 21:57:13 INFO FileInputDStream: New files on time 1421944033000 MS: 15 / 22/01 21:57:13 Information JobScheduler: Added jobs for the time 1421944033000 MS 15/01/22 21:57:13 Information JobScheduler: 1421944033000 ms.0 from the working time of work start work 1421944033000 MS 15/01 / 22 21:57:13 Information SparkContext: Starting work: StreamingKMean.java:33 15/01/22 21:57:13 Information: foreach on DAGScheduler: Job 3 End: foreach on StreamingKMean.java33, 0.0000 9 4 sentences Files collected from [] ------------------------------------------- 15/01/22 21:57:13 Junk JobScheduler: Job's Job Streaming Job 1421944033000 ms.0 From Time to Job Set 1421944033000 MS Time: 1421944033000 MS ----------------------- ---- ---------------- 15/01/22 21:57:13 Information JobScheduler: Job Streaming Job Started 1421944033000 ms.1 Time of Job 1421944033000 MS 15 / 01/22 21 : 57: 13 Information JobScheduler: Job Streaming Job Job 1421944033000 MS .1 Time 1421944033000 MS 15/01/22 21:57:13 Information JobScheduler Set: Total Delay: 0.028 for Time 1421944033000 MS (Execution: 0.013) 15/01/22 21:57:13 Info MappedRDD: RDD 5 Nick Releasing List 15/01/22 21:57:13 Information Blocker: RDD Removal 5 15/01/22 21:57:13 INFO FileInputDStream: 1421943973000 Approved older files older than MS: 15/01/22 21:57: 13 Info FileInputDStream: Cleared 0 old files that were over 1421943973000 MS: 15/01/22 21:57:13 Information ReceivedBlockTracker: Removing batches ArrayBuffer ()
Try it with another directory and then copy these files to that directory while the job is running is.
Comments
Post a Comment