1) Whenever you finish a project the next one pops up right away. Is this completely random or does something influence it? Like what zone you're in, the last artifact you collected, or something.

Completely random. The only thing that influences what you get is your skill, but only indirectly. Your skill determines which items are available to be chosen, but the choice is still random.

2) If you are working on a project that needs for example 32 artifacts, what happens if you continue to pick up artifacts when the bar says 32/32? Are those artifacts lost or are they stored and used on your project? If it's the latter, is it then shown anywhere how many artifacts you have extra or is that 'hidden'?

Artifacts are never lost. If you are 32/32 and pick up more artifacts, you will show as 35/32 and start the next one with 3/32.

3) I see people talking about buying artifacts, is that possible? I don't see them anywhere as an object so I wonder how to sell them.

You can buy key stones, which are items specific to each race which are equivalent to 12 fragments. These drop randomly when you are digging. These are things like Highborn Scroll, Troll Tablet, etc.

It’s also possible that you’re referring to the final output. Once you click the Solve, button you will receive an actual item. Common items (greys) can be safely sold to a vendor. Blues can’t be sold- so if you no longer want it, I guess you’d have to destroy it. Epics (at least the gear) can be sold, but the prices aren’t that great, so you’re probably better off keeping it. All of the non-common output from this profession are BOP, so they can’t be bought/sold on the AH.

I can semi answer 2 and 3.
2 if u get a 32 artifact project and u get more. if u go on your profession fr that artifact. you should 32+ (however many extra) when u solve it you WILL get the extra artifacts, that were not used for that one. In other words. u can collect as many as you want, and just solve none of your artifacts will be lost.

3. as for buying artifacts. im not sure but i think what your talking about is specific tablets.
certain artifact projects have a key on them. occasionally you will discover a, say troll rune.
that troll run can be used for a artifact project.
Ex. If you are working on a project that requires 32 artifacts. and you only have 19artifacts and a rune. you can use the rune to fill in a certain amount of artifacts for that rune.
So basically they are fillers for when your low on artifacts.

1) As I understand it, its RNG for the most part. Having a higher skill should help your chances of getting a rare artifact, but I've seen people get rares at 175, while I didn't a single one until 240. There are also a handful of rare items you can't get until 450. Ideally though, you should be repeating common items until you have all of them, though I've had a couple times where I've repeated common items without having the rest of them. Essentially, its entirely random.

2) Your fragments never go anywhere, so feel free to go over that 32 limit, the extras will start counting for your next project. I know a lot of people who save up troll fragments, as the only rare troll piece is a raiding-quality 2h sword that you can only see when you hit 450 skill.

3) The only thing exchangeable in archaeology currently are keystones. Highborne scrolls, troll tablets, dwarf rune stones, those kinds of things. Prices on these vary as (in my experience) some keystones are rarer than others. I've got a whole stack of troll keystones, but very rarely do a find the ones in Outland or Northrend, so prices tend to vary wildly from 40g to 500g.
The artifacts you get are stored permanently in the archaeology interface, so you might as well just vendor your finds (sadly they don't sell for much. at all.) unless you have bag/bank space to spare.

Hot damn I need to start typing faster. /shakesfist

once you get to 450+ the greys start vendoring for 10 gold or more. some of the blues are vendorable, and are pretty useless(looking at you arrival of the naaru)

Thank you all for the answers. It helps a lot and I appreciate the detailed answers.

Regarding my third question, I was indeed referring to keystones, but I didn't know what they were called or that they even existed :)