Online PCB Submission Service - Development Update [2/8/05]

Archive BatchPCB
1

Hello all, just an update on whats happening with the online DRC Service.

Over the last few days i have been writing the backbone systems to handle user data and backend database calls.

I had planned to sit down and blitz the whole system today and get the database structures in, finish the template systems and test the actual Design Rule Checking modules for any bugs.

BUT the unfortunate events in London this morning have kinda diverted my attention from the job. We are still get news updates comming in, hours after the incident. It has been a hectic week, Live8, G8, Winning the 2012 Olympics and then this!

I will be having today off, although there are some other ‘sparkfun’ related projects i am working on.

More news tomorrow,

  • Martyn
2

Are you located in Great Britain?

3

yup. Know a lot of people who know people that have been affected. ie. caught up in the stations when it happened.

Anyhow, working out bugs in the HTML ATM. Need it to pass accesibilty tests!

4

Now that I know where you are from, the little voice in my head switches to a British accent when I read your posts! :lol:

I’m about to give your online DRC a run for its money with a small but tightly packed PCB.

5

lol, the size estimator will take lots of time, it takes each point and compares it 2 or 3 times ;). lol.

6

I’ve been trying to submit my board online, but keep running into a little snag. After I submit my .drd file, all checks good, but then it asks you to register to complete the online submission part. I fill out the info, but never receive an email with my user information or the user id, which I need to finish up the submission. Any ideas…? Have tried it 3 seperate times, in case you were wondering :wink:

Thanks.

7

opps lol, the system on there is a ‘development’ release, which is not working properly.

You can use the links on the start page (bottom right) a new version is on the way, promise, working on it now.

8

pittuck:
Hello all, just an update on whats happening with the online DRC Service.

One thing I’d like to see in the on-line DRC which shouldn’t be too hard but would probably help catch many submittal mistakes would be to have users submit a ZIP file and have the system check the drill-file entries to see if there are corresponding pads on the top and bottom layers.

If there are 170 holes and all of them correspond with pads on the top and bottom layers, odds are pretty good that everything is aligned properly and nothing is mirrored. On the other hand, if all 170 of the holes correspond to pads in the top layer but only 23 holes correspond to pads in the bottom layer, that would suggest that the bottom layer was mirrored. If only a few (or none) correspond with pads on either layer, that would suggest that the NC drill file and Gerber files used a different origin, or format, or something.

This type of check would probably catch far more problems than could any check that simply examined a layer in isolation. Further, from a computational standpoint it should be very straightforward and easy to perform. What do you think of the idea?

9

I’m all for it. It’s much better than the existing check which gives you a very misleading ‘pass’ result even when the format is incorrect.

10

BigRedBee:
I’m all for it. It’s much better than the existing check which gives you a very misleading ‘pass’ result even when the format is incorrect.

Oh, great!!! :x

Well, I guess I’ll have to wait and see how my first board submission went then…fingers crossed

11

I’m sure your OK. If the drill files don’t line up w/ your board, it’s pretty obvious, and I’m sure your design would be rejected.

12

the problem is with drill alignment, there are many many ways to make pads. For example you can use many shapes or sizes which will all work. And even ‘incorrect’ files can pass Sparkfun DRC, these would always give errors in the system.

Finally, the amount of server load doing this sort of check is not tollerable. At the moment a 6k drill file took 7.08 seconds to process using the size checker (which does X/Y checking etc.), compared to the 0.21 seconds on my Laptop and 0.68 seconds on a dedicated server.

Basically sparkfun uses virtual hosting, loads of websites on one server. There is nothing wrong with this, it saves time, money and the need for experience in webserver management. But for CPU intensive data handling, this is not too easy, someone taking advantage of this could cause sparkfun.com to be closed down as a breach of AUP.

There have even been hacking attempts by some people uploading files to the server… None work, too good for simple problems :wink: lol j/k

13

I can certainly see how that’s a problem. Here’s a couple of ideas, not sure how valid they are.

  1. provide an ap that the user runs in HIS computer. Can’t one determine the board size from the component layer, then ensure that all drills are within that bundary?

  2. Change your import tools to tolerate a wider variety of drill file formats. I noticed that freedfm.com doesn’t see to care which format the drill files are in.

  3. provide more specific documentation and examples so the user can figure this problem out for himself. Either in the way of clear examples, or viewer tools and scripts that we can run. It seems possible to document the correct settings in ‘viewmate’, then import the drill file. Bad Drill files are very obvious.

There is some confusing/conflicting information running around these forums regarding leading / trailing formats that doesn’t help. See this thread http://www.sparkfun.com/cgi-bin/phpbb/v … php?t=1350.

One other problem is that sometimes the drill files are manually fixed on SparkFun’s end, leading the submitter to believe their format was correct. Later, that same format may be rejected, leading to more confusion.

One last comment. “Leading zero suppression” is the same as “omit leading zeros” is the same as “leading” when it comes to descriptions of the correct format.

Just my thoughts.

14

There is / was work under way on a proper DRC program that will run on Windows.

Allowing for a wider variety of drill files will defeat the means, specifically with regards to the X and Y co ordinate definitions which are ‘optional’. Sparkfun want both X and Y co ordinates, but can accept X or Y (if one is the same as the previous co-ordinate definition). The drill files checker does allow for this at the moment, but to check the drill holes and pads etc. it would mean pretty tightening the parameters.

I plan to get some definite examples from sparky as soon as the DRC is complete.

if you check now the latest version of the DRC is online, it allows some basic data entry (will be expanded in the next few daysto include more input on the boards) and the drill file checking works 80% (there are a few problems with drill number identification for when the users are trying to debug errors) and all the loop backs are in place for error / resubmission.

It took longer than expected to get the files on the server this morning, it should have taken 5 minutes to copy the files and check the settings / security bits. But due to some configuration differences between my test bed and the sparkfun server, it took over 2 hours. My timetable for today was shot to pieces! lol

Anyhow, i am working at it. Sparkfun it not my top priority as i am doing it fairly pro-bono when u look into time working on it and the amount i am getting paid! lol, no complaints tho.

One more thing, the FreeDFM system is nothing like what i am doing, they have a fully blown DRC program, probably developed by a professional program in VC or VB, the web front end just acts as a file uploader, there is a lot of processing that goes on behind the scenes!

15

Ok yesterday i corrected 2 bugs in the drill file checker which reported that a drill called T1 was not used, even tho it was! problem was the main body of the file called T001. All fixed now.

Also spent an hour tracking a ‘bug’ which was actually in my gerber layer file! the CAD software defulted to top copper layer on supposedly silk screen items which did not have the silk screen layer (top/bottom) specified. Wierd really, but took a while to get it working!

16

pittuck:
Allowing for a wider variety of drill files will defeat the means, specifically with regards to the X and Y co ordinate definitions which are ‘optional’. Sparkfun want both X and Y co ordinates, but can accept X or Y (if one is the same as the previous co-ordinate definition). The drill files checker does allow for this at the moment, but to check the drill holes and pads etc. it would mean pretty tightening the parameters.

I don’t really see the problem. Step 1 is to read the drill file into an array with drill size, X and Y coordinates in integer mils, and two “check-off” bits. Handling of numerical formatting and optional coordinates should not be difficult in this step.

Step 2 is to sort the drill file by X coordinate, subsorting by Y.

Step 3 is to read the top layer Gerber and, for every coordinate in the Gerber file, search the drill array for it (should take a reasonable length of time if the drill file is sorted); if found, mark the first “check-off” bit.

Step 4 is to do the same thing with the bottom gerber, but use the second check-off bit.

Step 5 is to read through the drill file and count how many entries have both, one, or neither check-off bit set. Then identify the sizes of all bits which don’t have both check-off bits set.

Shouldn’t be too hard to run the entire procedure in a reasonable length of time; as noted, it would provide excellent assurances that the board layers are properly lined up.

17

but for two problems:

  1. gerber files wont contain a point for every drill hole. I just checked my current project, about 10/80 holes do not extrude from their line on one layer, so this will give wrong results unless i graphically analyse the files (something i have been working on, but is not going well).

  2. run time, go to http://www.sparkfun.com/~mpittuck/size.php and upload and run a copper file. That does Step 3 (4) on one file. I tried it on a largeish design, took just under 30 seconds to process the file to give the following result:

X3.911 : Y0.126 => 0.492786 = 1 inch2

[The size checker is not complete BTW, we are not sure if it can reliably be used]

Now comes the problem, even with a reasonable size file, 5 to 8 seconds. For 2 files, max of about 13 seconds? (assuming one layer has more info than the other) processing of the drill data will take about 3 seconds as i dont currently use any of the data in the drill checker and possibly another second or 2 for cross referenceing.

18seconds for a ‘reasonable’ file. much more for larger layer files (the ones i tested was 335k giving about 30 seconds and 28k giving 7 seconds) So with large designs your going to need a minute, more? thein lies the problem, php will time out after 30 seconds.

Acceptable Use Policy of nearly all hosting companies will class more than 30 seconds processing time as being too much and will not change their server, the only way to get past it will be to get a dedicated server costing probably 4x the amound sparkfun pay at the moment per month.

I would love to have a all singing all dancing progam, i am sure i can do all that you ask, but i am unable to make it work within the brief i have been given. And something of that scale is going to cost a lot more than what i am asking for as payment.

Just so you realise, i am a semi-professional programmer, freelance. In monetry terms sparkfun will have paid me for about 6 hours work on my usual hourly salary, i am doing this as a favour / for experience.

For the moment i will work with what i have already done, i might be able to add new features at a later date. The plan is now to get the PCB Wizard complete so that when sending the final email to sparkfun there is a copy / paste section of text to put in the email which will help sparkfun with the common problems of format identification and incorrect board sizes.

18

pittuck:
but for two problems:

  1. gerber files wont contain a point for every drill hole. I just checked my current project, about 10/80 holes do not extrude from their line on one layer, so this will give wrong results unless i graphically analyse the files (something i have been working on, but is not going well).

In the PCB programs I’ve used, one creates a hole by creating a pad or via. Although a hole with a size of ‘zero’ won’t generate anything in the Gerber file, instructing users not to use a hole size of zero (use a size of eight if a copper-less hole is desired) shouldn’t seem too hard. Alternatively, one could report the number of holes without top and/or bottom pads and let users decide if it seemed reasonable for their application.

  1. run time, go to http://www.sparkfun.com/~mpittuck/size.php and upload and run a copper file. That does Step 3 (4) on one file. I tried it on a largeish design, took just under 30 seconds to process the file to give the following result:

Is the time CPU time, or is that just time to perform the physical upload [which may be substantially reduced if uploading a ZIP instead of a Gerber] If the former, what’s your server running on–an XT? I could write you a C program that would run on just about anything (including an IBM XT) and perform the three-file check in a second or two (well, maybe five or ten on an IBM XT) for boards of up to 32,000 holes; since a 10"x10" board with ten holes/inch in both directions would only be 10,000 I don’t think a 32,000-hole limit should be a problem.

18seconds for a ‘reasonable’ file. much more for larger layer files (the ones i tested was 335k giving about 30 seconds and 28k giving 7 seconds) So with large designs your going to need a minute, more? thein lies the problem, php will time out after 30 seconds.

What are you using to process the files? Trying to run something with Unix shell scripts would probably be too slow, but a decently-written C program should be pretty zippy.

Just so you realise, i am a semi-professional programmer, freelance. In monetry terms sparkfun will have paid me for about 6 hours work on my usual hourly salary, i am doing this as a favour / for experience.

Fair enough. I don’t know exactly what sort of platform the server’s being run on, but it shouldn’t be too hard for me to write a program that would take four arguments (drill, top, bottom, and “results” filenames) and produce a report of what seems kosher and not.

For the moment i will work with what i have already done, i might be able to add new features at a later date. The plan is now to get the PCB Wizard complete so that when sending the final email to sparkfun there is a copy / paste section of text to put in the email which will help sparkfun with the common problems of format identification and incorrect board sizes.

Well, if having a C program to do some validation would be useful, I might be able to help you out.

19

Again, sparkfun will have to get a dedicated server to run the DRC to use a C program.

These are not file upload times, but processing times. Remember one server will hold between 50 and 100 websites, and sparkfun is quite a hitter on bandwidth, so its a stressed server.

Anyhow, here are some problems: a section of my circuit

http://www.sparkfun.com/~mpittuck/tmp.GIF

The lower line was added for this example, its just a regulator i sometimes use. There is one pad which will give errors.

Anyhow, i will not be adding any more features to the system until i get a working system for sparkfun.[/img]

20

Ok i have uploaded a new version.

This one has layer checking for 2 layers, i will add some routines to handle pcb’s with only one layer later. I am goign to make a size calculator tutorial where u put in the width and length of the pcb and click go and it gives you the billable amount.

I could use the current calculator, but there are problems as there are not actual gerber definitions of the PCB size. So overhanging components and lack of silkscreen boarder will mean its wrong. I can leave it in as a option, any ideas?

The current tutorial involves printing and measureing the design, i know for me i print every design before i submit for PCB’s to be made to make sure all the component footprints are correct (for the components i have to hand).