如何才能比Next更快的访问RecordSet的记录?此问题已经解决,与各位FW共享一下心得! (300分)

  • 主题发起人 主题发起人 doxpix
  • 开始时间 开始时间
D

doxpix

Unregistered / Unconfirmed
GUEST, unregistred user!
比方我从数据库得到一个返回的recordset有6万条记录,如果我们要读取这6万条记录
通常是用not eof...next来逐条遍历记录,这样处理起来比较慢.我跟踪发现next耗费了
大量的时间,因为next会移动数据集的游标,实际上我并不需要移动游标.如果能得到内存
中每一条记录的指针就好办了!!!

大家有没有比较好的方法贡献一下?
 
你以为是记录文件啊,呵呵,
只有从光标方式,种类缓存等处进行优化,别无他法,
 
to kkyy:
没有用的,所有的记录都在内存中.
 
那你用开放源码的数据库接口读数据然后自己知道数据在内存中的存放格式就可以了,
 
就象SyBase的OpenClientLib 你用它取出数据然后就存为自己想要的格式,
 
还不如找个好点的数据库驱动更现实.
 
getrows方法.在微软的MSDN上面有说.
 
可以考虑用书签呀,书签就是指针
 
你不妨试一试使用计算字段,
计算字段的计算过程不用(并且也不能)显式的调用Next,
根据实践经验速度快多了,但使用起来可能麻烦些。
 
刚才看了看MSDN的GetRows,有这么一段:
After you call GetRows, the next unread record becomes the current record. If there are no more records, ADOCE sets the EOF property to TRUE.

那么这跟Next的速度应该差不多吧.

注意到adoquery有个ActiveBuffer属性,是PChar类型的,可能有戏!
 
设置合适的游标类型
 
to zqflying,
的确.计算字段不会移动游标,可惜我改成计算字段比较麻烦.
 
那就没有办法了,想提高效率当然要付出代价,
计算字段只能按顺序处理记录,最适合于需要遍历记录的情况,
使用Delphi的话估计没有比这更快的方法了(?不知道嵌入汇编咋样),
做起来可能麻烦些,但也总比自己再重新写一个优化内存处理的程序容易多了。
 
服务端游标,移动(Next)比客户端游标快很多很多。 (最适用于Access)

我的测试结果(当然把能优化的地方也都尽量优化了):10万条纪录,15个字段,全部访问一遍,7秒钟。
 
新思路!
可能使用MemoryTable是个好办法,让我先试试再说.
 
试用了几个内存表控件,速度没有明显改善.
发现移动光标(next)耗费了较多的时间,另外试用field[index].asinteger/asstring
是类型转换也花费了较多的时间.

那么只有直接访问内存块的方法了.
 
一些优化技巧(老生常谈了)
1.dataset.DisableControls;
2.界面控件使用BeginUpdate/EndUpdate;
3.如果只使用Next访问记录的话用ctOpenForwardOnly(仅前向)光标.
 
哈哈哈哈,踏破铁鞋无处觅,得来全不费功夫!
无意中在ExEx浏览到一贴,速度超快.
http://www.experts-exchange.com/Programming/Programming_Languages/Delphi/Q_20123177.html
摘录如下:
-----------------------------------------------------------------------------------------
Comment from heathprovost 05/24/2001 09:01PM PST

Your speed problem has nothing at all to do with your query string or
and settings there of. It is a problem with the way you are using
the ADOQuery component AFTER you run your query. This is a VERY
common mistake that people make when using ADOExpress components.
The purpose of the ado components is to allow Delphi programmers
to treat an ADO recordset as a TDataSet so that they can bind data
controls to it. This is of course very useful, but ALOT of stuff
is going on under the hood to allow this to happen. Alot of VERY
slow stuff. If, however, you dont need to bind to data controls,
you should never use the properties of an ADOQuery or ADOTable directly.
Instead you use the recordset property of the ADO component
and make direct ADO calls to do your work. What you had like this:

qrtest.first ;
with QrTest do
begin
slStoreData := TStringList.Create ;
while not EOF do
begin
slStoreData.Add(qrtest.Fields[0].Value);
next ;
end;
end ;

should be rewritten like this (add the ADOInt unit to your uses clause):


var
//these two vars will be used to cache the ADOQuery props
rs: _Recordset;
fld: field;

....

rs := qrtest.recordset; //set recordset to cache var
fld := rs.Fields[0]; //set desired field to cache var
//now use only methods of rs and fld
rs.MoveFirst;
with rs do
begin
while not EOF do
begin
//need to use vartostr() to make this safe
sl.Add(vartostr(fld.Value));
movenext;
end;
end ;
//clear the cache vars by setting them to nil
rs := nil;
fld := nil;

The code you gave took about 18 minutes on my computer.
The code above only took 6 seconds :)

Heath

ps: if you need to output more than 1 field, just create a cache var for each field before looping the recordset.


Comment from heathprovost 05/24/2001 09:04PM PST

btw, i created a test table to do my analysis.
It was on MS SQL 2000 and contained 5 fields and 236000 records.
That is what my timings were based on. Your timings may vary a bit but it shouldnt matter a whole lot.
-------------------------------------------------------------------------------------------------------
 
oh,核心就要用原生的ADO接口,ADOExpress控件组确实有很多问题。
 
后退
顶部