76 minutes
ago: Doing some maintenance on the server today. If you've placed an order and it hasn't processed yet, hang in there. Should be ready soon. I've got the car up on the jack and the tires are off...
Dismiss
Hi Guys, More questions regarding MS Access and SQL Server. Moving my application backend to SQL Server as first step, then completely web based as a second step. I have a procedure where we update/import 32,000 new products on a monthly basis. Originally this ran as a local procedure and took about an1 hour to run. I have attempted to rewrite this to work with SQL Server tables using stored procedures but now it takes 6 hours. Is it possible to write code that runs completely on an SQL server in the background? How do you loop through records on an import table, analyze the current record, and then import or pass on the record EFFICIENTLY in SQL Server. Please don't say "use local tables", because this is only the first step and the second step is to move the application completely to the web, so we have to make it work "up there".
Thank you
Kevin Yip
@Reply 13 months ago
All codes that run on SQL Server stay on the server. But are your data on the server too, are they local data that have to be uploaded to the server first? If you upload 30k rows to the server, that is never going to be fast. All web apps must be designed to make *minimal* number of reads and writes and minimal data per read and write -- because Internet speed is still nowhere as fast as local speed. Reading/writing 30k rows may be okay as a one-off. But if it's a regular occurrence, you need to change your design.
Yeah, you definitely do not want to be pushing 32,000 records one at a time. Access will happily sit there and grind away, but it will take hours, like you said. The problem is not really the amount of data, it is the ODBC and database overhead of opening and closing all those individual transactions. That overhead adds up fast and kills performance long before the raw data size would.
What I usually do is build a single pass-through SQL statement and let SQL Server handle it. That way the data moves in bulk, not piecemeal. If you are working with tens of thousands of rows, break them into chunks of a few thousand each so you are not building one monster string that chokes memory, but still avoid the row-by-row overhead.
This is one of the big themes I will be covering in my upcoming Access-with-SQL-Server series. Access is great for front-end forms and reporting, but when it comes to large updates, let SQL Server do the heavy lifting. That is how you get speed.
Kevin Yip
@Reply 13 months ago
I would put all the tables that receive heavy periodic updates in a separate database on SQL Server. Whenever I have a new version of those tables, I just upload the whole database to SQL Server on Winhost. That is much less resource-intensive than running queries to do the updates.
Sorry, only students may add comments.
Click here for more
information on how you can set up an account.
If you are a Visitor, go ahead and post your reply as a
new comment, and we'll move it here for you
once it's approved. Be sure to use the same name and email address.
This thread is now CLOSED. If you wish to comment, start a NEW discussion in
Access Forum.