If the motherboard has an AGP slot, then it doesn't have on-board video. However, I just finished working on a Dell Dimension 4100 with dead video. Ultimately discovered that the AGP card wasn't dead, the AGP slot was dead. Bought a mid-range ATI card that fits the standard slots and it works. Of course, it'll never be suitable for super fast or high-end graphics, but for normal uses it'll chug along okay.
Dell's motherboards aren't the greatest. I suspect you may have a bad slot. Although -- if you're getting any video at all, I guess that means the slot can't be dead.