c# - Linq to SQL vs Serialization -


I have some tables in the MSSQL database, each has about 5-10 properties, there are some simple organizations among tables, But there are 500,000 to 1,000,000 rows in each table.

There is an algorithm that runs on that data (all), so before running the algorithm, I have no algorithm data to retrieve all the data from the database, only reads it, so I need to recover the data.

I am using SQL from SQL. It takes about two minutes to get all the data. What I want to know is whether the serialization file is and then the deserialization (when necessary) will actually load the data fast.

The data is about 200 MB, and I have no problem saving it to disk. So, would it be faster if the objects were dasarized from the file or using LINQ 2 SQL DataContext?

Any experience with this?

I will argue that LINQtoSQL may not be the best option for such an application. When you are talking about so many objects, you get some overhead to create an object instance (your constant class).

I will choose a solution where a stored procedure is only the ado. Net receives required data The application stores it in memory (memory is cheaper nowadays, 200 MB should not be a problem) and analysis algorithm is run on in-memory data.

I do not think you should store the data on the file. Finally, your database is one or more files that are read by the database engine. So you either

  • let the database engine read your data and you analyze it, or
  • let the database engine read your data, you write it in the file , You have read the file (the same data is reread, but now you do it yourself) and you analyze the data

In the later options, lots of benefits Overheads are included, as far as I can see. / P>

Edit: If your data changes very rarely at times, then you should first consider pre-processing data in the database (in the database or in the file system) prior to analysis and caching. can do. It only makes sense if your preprocessed data can be analyzed (much) faster than raw data, probably some preprocessing can be done in the database.


Comments

Post a Comment

Popular posts from this blog

python - Overriding the save method in Django ModelForm -

I'm having trouble overriding a ModelForm save method. I get this error I'm: Exception type: TypeErier Exception Value: Save () found an unexpected keyword argument 'committed' My intentions include a form 3 fields Submit several values ​​for, then create an object for each combination of those areas, and to save each object file models.py class collarsult type (models.Model): id = model. AutoField (db_column = 'icontact_result_code_type_id', primary_key = true) callResult = models.ForeignKey ( 'CallResult', db_column = 'icontact_result_code_id') campaign = models.ForeignKey ( 'campaign', db_column = 'icampaign_id') CALLTYPE = models.ForeignKey ( 'CALLTYPE', db_column = 'icall_type_id') agent = models.BooleanField (db_column = 'bagent', default = true) teamLeader = models.BooleanField (db_column = 'bTeamLeader', default = true) active = models.BooleanField (db_column = Django.form...
Read more

html - CSS autoheight, but fit content to height of div -

I have a div in which there are three children- div in the left and middle div: float: left while true: float: true because it is my layout Actually will spoil, I've used ClearFix Hack: .cf {zoom: 1; }. Cf: First,. Cf: {content: ""; Display: Table; }. Cf: After {clear: both; } It still works, but I want to be the right div an indicator. So it should fill 100% of the parent unit of height. How can I complete this? PS is the full code: & Lt; / Div & gt; & Lt; Div class = "mobile_content" & gt; & Lt; H5 & gt; {{Data.titel}} & lt; / H5> & Lt; Table & gt; & Lt; TR & gt; & Lt; Th & gt; Komponist: & lt; / Th & gt; & Lt; TD & gt; {{Data.komponist}} & lt; / TD & gt; & Lt; / TR & gt; & Lt; TR & gt; & Lt; Th & gt; Instrument: & lt; / Th & gt; & Lt; TD & gt; {{Data.instrumente}} & lt; / TD & gt; & Lt; / TR & gt;...
Read more

qt - How to prevent QAudioInput from automatically boosting the master volume to 100%? -

I have been trying to use Qt5 multimedia to record audio with QAudioInput, however, when I see If QAudioInput is started, it increases the master volume of my sound device to 100%. How can I prevent QAudioInput from changing the master volume? My current development platform is Linux with PalsAdio (with flat audio disabled). How can I use QAudioInput : QAudioDeviceInfo device_info = QAudioDeviceInfo :: defaultInputDevice (); QAudio format format; Format.setSampleRate (44100); Format.setChannelCount (1); Format.setSampleSize (16); Format.setCodec ("Audio / PCM"); Format.setSampleType (QAudioFormat :: SignedInt); Format.setByteOrder (QAudioFormat :: LittleEndian); Std :: cout & lt; & Lt; Device_info.deviceName (). ToUtf8 (). ConstData () & lt; & Lt; Std :: endl; QAudioInput * default_device = new QAudioInput (device_info, format); QIODevice * default_io_device = default_device-> Start (); a QAudioInput.setVolume () method is not you I ...
Read more