phpBB2 attachment support for ImpEx

[OmerE]

Hi,

This "mod" will import attachments from your phpBB2 board to
your vB3 board, and comes in the form of another ImpEx module for
the phpBB2 system.

You must patch your impex before you start the import process.
You cannot use this to import attachments from a phpBB2 board
to an existing vBulletin board already imported, as crucial data is
missing (namely, mapping the old phpBB2 postid to the new vB postid).

It also requires a small modification to one of the core ImpEx files
to fix a small bug (missing addslashes on a variable passed to the DB),
and several other phpBB2 module files.

I wrote this for personal use, in order to import my own board.
Later tested it on another board and fixed a few bugs, but I'm certain
there may be some left.

It's based on the IPB attachment importer, but I believe I'm allowed
to post it here, as only registered owners should be able to grab the
code.

Two small comments - during the import stage, it will ask you what's
the largest attachment you want to import. This is limited by several
factors - the setting of the MySQL variable "max_allowed_packet" (as
attachments are imported into the DB), and the type of the filedata
field in the attachment table. vB uses "mediumtext", which will
limit you to 16MB attachments. If you want to import larger attachments,
be sure to change these two. If using the slow queries log, you may
also want to consider either disabling it temporarily, or raising the
long_query_time value - otherwise the MySQL will choke itself just
trying to write out almost every single large attachment to the logfile...
which will slow things down even more.

Finally - in order for the import process to finish quickly and efficiently,
an INDEX is required on the importpostid column in the "post" table.
I did not add this automatically, as I'm not sure where all those added
fields that the importer adds to tables are cleared up. Before running
the attachments module (but after running the posts module), execute
this query:
ALTER TABLE `post` ADD INDEX `importpostidx` ( `importpostid` )

This will greatly speed up the process (running without it would
be much, much slower - try to avoid doing that).

I dunno if I'm allowed to post the modified ImpEx files themselves,
so I'll post a patch file from ImpEx 1.21 (much easier than listing by hand :) ).
If the files can be posted, please let me know, and I'll attach those as well.

To use the patch file put it one level above the "impex" directory,
then run:
patch -p1 -d impex < phpBB2_attachments_import.patch

EDIT: forgot to include the importattachmentid column here.
Look 2 posts down for instructions.

Hope this helps someone.

[rookie7]

This is awsome. Been wanting to move my phpBB attachments. Now I finally can. Will surely give this a try.

Thanks for sharing.

*install*

[OmerE]

In the patch file included here I neglected to add one required line
which is required in order to create a temporary column in the attachment
table during the upgrade process.

In impex/systems/phpBB2/001.php, around line 89, and AFTER you
use the patch file, find this:
'13' => array('post' => 'importpostid')

Replace with:
'13' => array('post' => 'importpostid'),
'14' => array('attachment' => 'importattachmentid')

If you're in the middle of the process, just add a BIGINT colum
named 'importattachmentid' to the attachment table.

[patj]

Can you clarify on what you mean by adding a BIGINT column to the attachment table? I have a phpBB import already underway and successful. I've patched my impex with both sets of changes but the attachment option does not show up in ImpEx.

Thanks in advance.
-Pat

[Dean C]

I'll forward this over to Jerry to confirm this is allowed :) Also he may have some use for it.

[patj]

Cool. To recap - I've already done an import of my users/messages and that worked OK. I ran patch, and replaced the missing line. When I fire up ImpEx from the ACP, no option shows up to import phpBB attachments. I'm wondering if I have to bump a file to tell it to restart the import.. or whether I need to extend the SQL schema with a new column as indicated above.

Your help is appreciated. Have a nice weekend.
-Pat

[cecilBergwin]

Hello,

Ive currently installed a fresh vB3 and imported all my data from my old phpbb forums. My problems is now to import the attachments as its the only thing left for me to do.

Ive downloaded the "Attachment Fix" script but im a little miffed on where or what todo with this. Could somebody point me in the right direction as i would dearly like to import all my attachments and everything upto now has gone really well.

From a Board already up with only attachments waiting todo, what will i have todo in order to execute this script and import my phpbb attachments.

Any help greatly appreciated folks, ill be eyes open Thx.

[audigy]

i'll help ya out...where are you stuck? After you did the mod there should be a module that says Import Attachments in ImpEx UNLESS you added this mod AFTER you started the inital import.

[cecilBergwin]

Thx Audigy mate,

Well im not sure ive added the module, above in instructions it says

To use the patch file put it one level above the "impex" directory,
then run:
patch -p1 -d impex < phpBB2_attachments_import.patch

Ive placed that file in the correct location but then what ? its say run the patch -p1 -d impex < phpBB2_attachments_import.patch but im presuming this is from sever side ? as im not sure on how to execute this script at all.

Many thx aud, appreciate your reply/time.

Cecil

[OmerE]

As someone mentioned, the module will not show up if you only
added it after starting an import session, due to the datastore ImpExSession
(or whatever it's called,I forget) storing a serialized version of the current
import state (one would hope that sometime in the future it would actually
store errors as well, kind of useless right now).

Regardless, the attachment import requires that the posts import module
adds another field to the table, so even if it did show you'd need to import
everything all over again.

As to how to use this: since I did not want to give step by step
instructions on how to patch the ImpEx files, I included a patch file.

If you do not have command line access to the machine your vB3 is running
on, you'll have to unpack impex locally, patch it, then transfer things over
to the server. If using Windows, grab the patch utility here:
http://prdownloads.sourceforge.net/gnuwin32/patch-2.5.4-3-bin.exe?download

Or simply install Cygwin.

Hope this helps.

[cecilBergwin]

Thx very much, ill be looking into this straight away.

Cheers gents.

[patj]

OmerE,

Kindly edit your original post, and include that it should be installed BEFORE importing your phpBB settings. This has left me in a bind, and I will sit without phpBB attachments for now as I already have 60 users online.

Thanks for contributing your efforts.

Pat

[touristz]

Can anyone please help me to patch the files? I don't have Linux, and I tried both GnuWin32 and Cygwin.

Using Cygwin I got this error...

patching file ImpExDatabase.php
patching file systems/phpBB2/000.php
Hunk #1 FAILED at 14.
1 out of 2 hunks FAILED -- saving rejects to file systems/phpBB2/000.php.rej
patching file systems/phpBB2/009.php
patching file systems/phpBB2/013.php

[OmerE]

Can anyone please help me to patch the files? I don't have Linux, and I tried both GnuWin32 and Cygwin.

Using Cygwin I got this error...

patching file ImpExDatabase.php
patching file systems/phpBB2/000.php
Hunk #1 FAILED at 14.
1 out of 2 hunks FAILED -- saving rejects to file systems/phpBB2/000.php.rej
patching file systems/phpBB2/009.php
patching file systems/phpBB2/013.php

Are you using the latest ImpEx?
Post the rejects file (systems/phpBB2/000.php.rej), it will show
what lines caused it to abort patching.

Don't forget to enclose within code tags.

Omer.

[touristz]

Thanks for the response. But I managed to patch the file manually using notepad. Took awhile for me to understand how the patch file works :D

Thanks again! It worked flawlessly!

[Paul R]

Hi OmerE,

I got the same msg as touristz on my Linux box.

Here is the lines from the rej file.

***************
*** 14,20 ****
*
* @package ImpEx.phpBB2
* @version $Revision: 1.30 $
- * @author Jerry Hutchings <jerry.hutchings@vbulletin.com>
* @checkedout $Name: $
* @date $Date: 2004/06/28 18:28:32 $
* @copyright http://www.vbulletin.com/license.html
--- 14,21 ----
*
* @package ImpEx.phpBB2
* @version $Revision: 1.30 $
+ * @author Jerry Hutchings <jerry.hutchings@vbulletin.com>, minor additions
+ * by Omer Efraim <oefraim@gmail.com> to support attachments
* @checkedout $Name: $
* @date $Date: 2004/06/28 18:28:32 $
* @copyright http://www.vbulletin.com/license.html

The rest of the file were patched sucessfully.

Thanks for taking the time to look at this. Would love to be able to get the attachments over from my old phpbb2 board

Paul

[OmerE]

Hi OmerE,

I got the same msg as touristz on my Linux box.

Here is the lines from the rej file.

...

The rest of the file were patched sucessfully.

Thanks for taking the time to look at this. Would love to be able to get the attachments over from my old phpbb2 board

Paul

Hi,

Since patch should be patching the files hunk by hunk, it should be fine.
To be certain, take a look at 000.php - if you can find this text in it
"function get_phpbb2_attachment_details" - then all is well.

Omer.

[Jerry]

If anyone would like to offer the board and attachments dir for dev I'll include an attachment module in the official importer, its just to date I've not got a phpBB site with the attachments, I've only ever got the mysqldump.

PM me over at vb.com.

[tide]

Thanks so much for this patch. If not I would have had to write it myself. Good work!

[mels]

Hello there,

So great to hear this excellent news. I went ahead and tried to import the attachments. We ran the patch and got a couple of errors which I'll show later in this post.
I do see the import attachments module in my impex list and I can run it. Except it says on each line "skipping"... and sometimes it does say "skipping, possible PM attachment" and then it shows the attacht ID but when the module is over and I'm redirected to my impex main page, it's says 0 imported, 0 failed. What am I doing wrong?
Also, here's the info from the patch :

From 009.php.rej

***************
*** 116,122 ****
$start = time();

$post_object = new ImpExData($Db_target, $sessionobject, 'post');
-
foreach ($posts_array as $post_id => $post)
{
$try = $post_object;
--- 116,121 ----
$start = time();

$post_object = new ImpExData($Db_target, $sessionobject, 'post');
foreach ($posts_array as $post_id => $post)
{
$try = $post_object;
***************
*** 124,129 ****
$try->set_value('mandatory', 'userid', $users_ids["$post[poster_id]"]);
$try->set_value('mandatory', 'importthreadid', $post['topic_id']);

$try->set_value('nonmandatory', 'visible', '1');
$try->set_value('nonmandatory', 'dateline', $post['post_time']);
$try->set_value('nonmandatory', 'allowsmilie', $post['enable_smilies']);
--- 123,129 ----
$try->set_value('mandatory', 'userid', $users_ids["$post[poster_id]"]);
$try->set_value('mandatory', 'importthreadid', $post['topic_id']);

+ $try->set_value('nonmandatory', 'importpostid', $post_id);
$try->set_value('nonmandatory', 'visible', '1');
$try->set_value('nonmandatory', 'dateline', $post['post_time']);
$try->set_value('nonmandatory', 'allowsmilie', $post['enable_smilies']);


From 000.php.rej

***************
*** 14,20 ****
*
* @package ImpEx.phpBB2
* @version $Revision: 1.30 $
- * @author Jerry Hutchings <jerry.hutchings@vbulletin.com>
* @checkedout $Name: $
* @date $Date: 2004/06/28 18:28:32 $
* @copyright http://www.vbulletin.com/license.html
--- 14,21 ----
*
* @package ImpEx.phpBB2
* @version $Revision: 1.30 $
+ * @author Jerry Hutchings <jerry.hutchings@vbulletin.com>, minor additions
+ * by Omer Efraim <oefraim@gmail.com> to support attachments
* @checkedout $Name: $
* @date $Date: 2004/06/28 18:28:32 $
* @copyright http://www.vbulletin.com/license.html


Thank you for your help!

Mels.


mmm... okay here's a little edit. Turns out my path was wrong ;) Now I get another error message though... which is :

"Imported pm_text, Error with importing attachment"

On each and every line that is treated by the script... Can anyone help me on this one ?
1. Why this error?
2. Why only pms ?? Where are all my posts attachments ? :ermm:

[OmerE]

I've received information about the issue from Melanie (off-board), and it
seems that the current ImpEx is compatible with the patch, but not with
the patch file. I'll post a new patch file as soon as I have some time.

Omer.

[zigx]

I've received information about the issue from Melanie (off-board), and it
seems that the current ImpEx is compatible with the patch, but not with
the patch file. I'll post a new patch file as soon as I have some time.

Omer.
is this updated already?

[zigx]

Hey guys, im getting this error:

Fatal error: Cannot redeclare get_phpbb2_attachment_details() in /var/www/domain/impex/systems/phpBB2/000.php on line 978

Any ideas?

[zigx]

for some reason that function and the one under it were in there twice..just deleted an instance of both of htem and that fixed it.

[zigx]

hi OmerE, check the attachment i put on thispost...

i imported everything fine until i got to the attachment module.... hit attachment, then the error below popped up and my browser memory usage started to SKY rocket until the computer froze...

Now when i try to go back into impex's importer area it gets stuck in a loop and trys to load the frame for infinity... not sure what to do.

i couldnt get more details than this Screen shot when it crashed, but do you have any idea whats going on? or anyone for that matter?! thanks a lot guys.

and especailly thanks for this hack because if i can get it to work....i will consider you my life saver lol.

[zigx]

In the patch file included here I neglected to add one required line
which is required in order to create a temporary column in the attachment
table during the upgrade process.

In impex/systems/phpBB2/001.php, around line 89, and AFTER you
use the patch file, find this:
'13' => array('post' => 'importpostid')

Replace with:
'13' => array('post' => 'importpostid'),
'14' => array('attachment' => 'importattachmentid')

If you're in the middle of the process, just add a BIGINT colum
named 'importattachmentid' to the attachment table.

also, my 001.php didnt have that line in it... but i added the BIGINT column to the end of the table.

[Jerry]

If you PM me over at vbulletin.com I will add this offically so you don't have to mess about with hacks.........

[Jerry]

also, my 001.php didnt have that line in it... but i added the BIGINT column to the end of the table.

It wouldn't all that has changed now.

[zigx]

wow im BRAND new to vB so this is all new to me... ill drop u a PM.

[Jerry]

wow im BRAND new to vB so this is all new to me... ill drop u a PM.

Well I'm one of the developers and have been after a board with attachments for a while so I can add the attachment module.

[Ericsson]

patch -p1 -d impex < phpBB2_attachments_import.patch
Were do I run this?

[Jerry]

patch -p1 -d impex < phpBB2_attachments_import.patch
Were do I run this?

You don't, its woefully out of date and the standard importer deals with attachments.

[Divvy]

Damn... can someone help us?
I using impex 1.90 last version... and I need to import attachments from phpbb 2.0.23
OmerE, HELPPPPPPP :)

[Jerry]

This mod is obsolete ImpEx has support attachments for years now.

[Divvy]

Hi Jerry, are you sure? And works with this phpbb2 mod by Acyd Burn?