Impex - extending beyond username, email, password for text import (CSV)

[codegrunt]

Howdy. I have a client that wants to import an existing CSV based user list which includes much more detail than what the default Impex text importer allows. Specifically, it includes custom profile field related data, address data, etc. while the default text importer only offers the 4 fields username, password, email and id.

Looking at the files associated with the text importer (000.php - 0002.php), this system does not actually work with full CSV files as it does not appear to have the concept of quoted strings or escaping. There are no calls to fgetcsv() for example. So contrary to some postings in the official forum, CSV format is not fully supported by this importer.

So it looks I need to roll a custom solution. Before I go off an adventure though I thought I would ask around to see if I can speed this process up.

My questions are:

1. is there a more fully fleshed out CSV based script(s) available for importing user data from CSV?

2. if not, are there any guides available covering how to approach a custom impex "system" file? Can I just copy "impex/systems/txt_file" to a new directory name and start editing or do I need to update all class definitions to have unique names, etc.?

3. do any of the board associated import scripts have a more fleshed out user import (including profile fields) that could be adapted for use with a custom CSV importer?

Any and all assistance here appreciated.

Cheers

[MOGmartin]

Hi Codegrunt,

I cant help you on your quest, but I would like to add that I am also stumped on this one...

Im trying to import 8,000 users from an older system, CSV is my only route - and the damn thing doesnt work.

I am also interested in getting someone to help out on this, even as paid work.

thanks

MOGmartin

--------------- Added [DATE]1284503466[/DATE] at [TIME]1284503466[/TIME] ---------------

I would also like to add into this thread, that impex -> txt files is an absolute piece of f*****g s**t app.

After hours of playing around, I finally got it to work by RE-SAVING my text files in notepad, rather than excel.

Finished the import, got my users, albeit with only 1/10th of the data I really need, and guess what ---- all my forums have disappeared.

After 10 hours of trying to get this working, Im now really, really hacked off with vB.

[codegrunt]

I ended up having to write something custom to get this working. My mod is pretty specific to my situation (predefined columns + some custom profile fields) but could potentially be adapted to others as long as we are just talking user related data and not forum related changes.

Essentially I cloned the txt import module and changed the class definitions to reflect the new name, rewrote the import process to use fgetcsv(), hardcoded the columns to the fields I was using and then hacked in some extra stuff to deal with the custom profile fields I needed.

I should note that that stock code for handling custom profile fields appears to be broken for 4.* as it assumes existence of a column called "title" in the profilefields table that does not actually exist. I worked around it by inserting data into the userfields table directly instead of relying on the Impex function for this purpose.

Anway, I was able to import 64 users without any obvious issues using my semi-custom version so it is possible to do with some work.

Cheers

[radiofranky]

Hi,
I would like to import csv to vb database(+10000 threads). i have column "forum id", "user name", "title", "post text", "post date". is there a sample sql scripts that can start play with?

[point4design]

I know I'm dredging up an old thread here, but I've searched and can't find an answer. I need to import from a CSV and I need to be able to import more than just the 3 basic columns of info. Does anyone have a script I can use for this?