parallel processing - How to parallelize topicmodels R package -


I have a document (~ 50,000), that I have converted to a corpus and unfortunately using LDA objects The module package in the subject, to test more than 150 subjects, it takes several hours.

So far, I have found that I can test many different cluster sizes together: 

  Library (theme) Library (PERIR) Library (Forch ) Library (DOMC) Register DOMC (5) # 5 Core DTM # My DoculatorMatrix Seek & Lt; - Use the Seek (200500, 50 = 50) model & lt; - llply (cec, function (d) {lda (dtm, d)}, .arrell = t)

There is no way to parallel the LDA function so that it can run faster (Instead of running several LDAs at one time)?

I am not familiar with the LDA function but say I have divided the fund into 16 pieces , And put each piece in a list named corpus16list .

In order to run it in parallel y, something like the following must be done: 

  library (doParallel) cl & lt; - registerDoParallel (cl) for makeCluster (16) # 16 processor # Now start chain nchains & lt; - 16 my_k & lt; - A Vector result with list # 6 or 16 elements & lt; - foreach (i = 1: nchains, .packages = c% dopar% {Results & lt; - LDA (corpus16list [[i]], K = my_k, control = my_control)}, .progress = " Text ")) Returns (Results)}

The result is result_list , which is a list, you can connect with 16 outputs out of 16 chains that fit you. Or use a .combine function in the forehead (which is beyond the scope of this question).

To send different values ​​of i to control , k or whatever you need to do.

This code should work on Windows and Linux, and with this how ever many cores you need.


Comments

Post a Comment

Popular posts from this blog

python - Overriding the save method in Django ModelForm -

I'm having trouble overriding a ModelForm save method. I get this error I'm: Exception type: TypeErier Exception Value: Save () found an unexpected keyword argument 'committed' My intentions include a form 3 fields Submit several values ​​for, then create an object for each combination of those areas, and to save each object file models.py class collarsult type (models.Model): id = model. AutoField (db_column = 'icontact_result_code_type_id', primary_key = true) callResult = models.ForeignKey ( 'CallResult', db_column = 'icontact_result_code_id') campaign = models.ForeignKey ( 'campaign', db_column = 'icampaign_id') CALLTYPE = models.ForeignKey ( 'CALLTYPE', db_column = 'icall_type_id') agent = models.BooleanField (db_column = 'bagent', default = true) teamLeader = models.BooleanField (db_column = 'bTeamLeader', default = true) active = models.BooleanField (db_column = Django.form...
Read more

html - CSS autoheight, but fit content to height of div -

I have a div in which there are three children- div in the left and middle div: float: left while true: float: true because it is my layout Actually will spoil, I've used ClearFix Hack: .cf {zoom: 1; }. Cf: First,. Cf: {content: ""; Display: Table; }. Cf: After {clear: both; } It still works, but I want to be the right div an indicator. So it should fill 100% of the parent unit of height. How can I complete this? PS is the full code: & Lt; / Div & gt; & Lt; Div class = "mobile_content" & gt; & Lt; H5 & gt; {{Data.titel}} & lt; / H5> & Lt; Table & gt; & Lt; TR & gt; & Lt; Th & gt; Komponist: & lt; / Th & gt; & Lt; TD & gt; {{Data.komponist}} & lt; / TD & gt; & Lt; / TR & gt; & Lt; TR & gt; & Lt; Th & gt; Instrument: & lt; / Th & gt; & Lt; TD & gt; {{Data.instrumente}} & lt; / TD & gt; & Lt; / TR & gt;...
Read more

qt - How to prevent QAudioInput from automatically boosting the master volume to 100%? -

I have been trying to use Qt5 multimedia to record audio with QAudioInput, however, when I see If QAudioInput is started, it increases the master volume of my sound device to 100%. How can I prevent QAudioInput from changing the master volume? My current development platform is Linux with PalsAdio (with flat audio disabled). How can I use QAudioInput : QAudioDeviceInfo device_info = QAudioDeviceInfo :: defaultInputDevice (); QAudio format format; Format.setSampleRate (44100); Format.setChannelCount (1); Format.setSampleSize (16); Format.setCodec ("Audio / PCM"); Format.setSampleType (QAudioFormat :: SignedInt); Format.setByteOrder (QAudioFormat :: LittleEndian); Std :: cout & lt; & Lt; Device_info.deviceName (). ToUtf8 (). ConstData () & lt; & Lt; Std :: endl; QAudioInput * default_device = new QAudioInput (device_info, format); QIODevice * default_io_device = default_device-> Start (); a QAudioInput.setVolume () method is not you I ...
Read more