[tateu]

Ok, so I've fixed the other UTF-8 problem that I found. If you want to see the error I am talking about, create an email with:

Ths ”is” jst an example for the spell checker.

For the quotation marks, don't use alt-k. Use the special curly ones found by clicking on the symbol button. They are UTF-8 characters. Now spell check it, using the original BBCorrectorServer.cgi or my first attempt at a fix from a few posts back. You should get an error on your Blackberry about malformed UTF-8. You can then try it using my latest modified version found below and, hopefully, it should now work.

It's kind of a strange hack, but it has worked so far. With PHP, I just needed one extra line of code, utf8_decode($text) which replaces the multibyte unicode characters with a single byte question mark, and everything works. With Perl, though, I couldn't get it to work the same. Perl has, what I thought was a similar function, utf8::decode($text2Check), but it doesn't do what I need it to and I couldn't find any other function to do it either. I am in no way a Perl or PHP expert, just a hobbyist.

What I ended up doing was to write the temp file in "ISO-8859-1" with the multibyte unicode characters written as a hexidecimal representation \x{022d}. I then open and read the temp file, replacing all occurances of \x{022d} with a single byte space character, then overwrite the temp file with the modified text. This modified text is then fed to aspell for checking.

As I said, this is a strange hack. It seems like there should be a way to convert the multibyte characters to a single byte space character before writing to the temp file (like the PHP function). This would save 2 steps of reading and writing the temp file again. I've found some Perl modules that seem like they might do the trick, but they are not installed on my internet server. I could install them locally on my intranet, but I don't have admin control over my internet server so, even if they did work, I couldn't use them with my Blackberry. I will continue to look into other, more elegant options, maybe a Perl expert with a Blackberry will eventually pass by here and fix it the right way. Until then, here is another modified version for people to try out:

#!/usr/bin/perl

#
# BBCorrector Server
#
# This script accepts a block of text from the BlackBerry software
# BBCorrector, runs it through Aspell, and returns an XML packet
# indicating spelling errors.
#
# NOTES:
#
# - It is recommended you protect this script with some form of HTTP
#   authentication. BBCorrector is setup to handle Basic Authentication.
#
use CGI qw(:standard Vars);
use File::Temp qw/ tempfile tempdir /;

# The Aspell executable
my $cmdAspellExe = "aspell";

# Language is US English
my $lang = "en_US";

# Options for Aspell. Puts Aspell into Ispell compatibility mode
# so that its output is written to stdout
my $cmdAspellOptions = "-a --lang=$lang";

my %FORM = Vars();

# Get the block of text from the HTTP parameter "check"
my $text2Check = "$FORM{check}";

# Convert line endings to a common format. In this case whatever
# line ending combination we get (CRLF, LF, etc), we convert to a standard
# format of LF
$text2Check =~ s/\x0D\x0A|\r/\n/g;

# Create a temporary file to store our block of text
my $dirTemp = tempdir( CLEANUP => 1 );
my( $tempHandle, $tempFilename ) = tempfile( DIR => $dir );

# Removed the following lines in order to reopen the file
# in ISO-8859-1 mode - JHH 2005-07-09
## Split block of text into lines and write to temp file
#@lines = split( /\n/, $text2Check );
#for my $line ( @lines ) {
#  # Force Aspell to check whole line via ^ contol character
#  print $tempHandle "^$line\n";
#}
close $tempHandle;

# Hack to get UTF-8 data from Blackberry to work with non UTF-8 version of Aspell
# Open and write to temp file in ISO-8859-1 mode - JHH 2005-07-09
open(F, ">>:encoding(ISO-8859-1)", $tempFilename) or $err = $!;
if ($err) {
  print header;
  print $err;
  exit;
}
binmode(F, ":encoding(ISO-8859-1)");
# Split block of text into lines and write to temp file
@lines = split( /\n/, $text2Check );
for my $line ( @lines ) {
  # Force Aspell to check whole line via ^ contol character
  print F "^$line\n";
}
close F;

# Continuation of UTF-8 Hack
# Open tempfile which now has UTF-8 characters unpacked to HEX in \x{02dd} format
# Since the UTF-8 chars don't need to be spell checked, replace each occurance
# of \x{022d} with a space
# This causes all of the larger UTF-8 chars to be counted correctly as 1 char
# - JHH 2005-07-09 
my @contents = '';    
open (LOG, $tempFilename) or $err = $!;#|| die "Can't open - $!\n";
if ($err) {
  print header;
  print $err;
  exit;
}
@File = <LOG>;
close (LOG);
for (@File) {
  s/\\x{[a-f,A-F,0-9]*}/\ /g;
  push(@contents,$_);
}

# Now rewrite the fixed output back to the temp file - JHH 2005-07-09 
open (LOG, ">$tempFilename") or $err = $!;
if ($err) {
  print header;
  print $err;
  exit;
}
print LOG (@contents);
close(LOG);

# Hack to remove malformed UTF-8 data that gets sent back to the
# Blackberry causing errors
# Change the @lines array to use the same data as the fixed UTF-8
# data rewritten to the temp file but remove the special ^ char
# from the beginning of each line - JHH 2005-07-09 
@lines = ();
for (@contents) {
  s/^\^//g;
  push(@lines,$_);
}

# XML packet has format such as:
# <spell-results>
#   <error>
#     <word>maan</word>
#     <position>12</position>
#     <suggest>Man</suggest>
#     <suggest>man</suggest>
#     <suggest>moan</suggest>
#   </error>
#   <error>
#     <word>helllo</word>
#     <position>33</position>
#     <suggest>hello</suggest>
#   </error>
#   <error>
#     <word>chris</word>
#     <position>41</position>
#     <suggest>Chris</suggest>
#     <suggest>Charis</suggest>
#   </error>
# </spell-results>
my $xmlPacket = "<spell-results>";

# Do this here so that when we are debugging we can display it in return output
print header;

# Keeps track of current line number
# Because of the UTF-8 hack, $linenum now needs to start at 1 instead of 0
# - JHH 2005-07-09 
my $lineNum = 1;

# Keeps track of the absolute position in the block of text
my $posAbsolute = 0;

# Execute Aspell
my $cmd = "$cmdAspellExe $cmdAspellOptions < $tempFilename 2>&1";
# TODO: $status most likely only tracks wether the fork failed or not, not
# whether the actual command we are running (ie: aspell) failed
my $status = open ASPELL, "$cmd |";

if ($status > 0) {
  # Parse Aspell output
  for my $cmdReturn (<ASPELL>) {
    chomp($cmdReturn);
#print "$cmdReturn<br>\n";

    if( $cmdReturn =~ /^\*/ ) {
      # Line begins with *. Do nothing.

    } elsif( $cmdReturn =~ /^(&|#)/ ) {
      # Line begins with & or #.

      # Start error element
      $xmlPacket .= "<error>";

      # Split return line up for easier access
      my @tokens = split(" ", $cmdReturn, 5);

      # Add word element which contains original misspelled word
      $xmlPacket .= "<word>$tokens[1]</word>";

      # Need to work out absolute position in file, not just position in current line
      my $offsetIdx = 3;
      if ($cmdReturn =~ /^\#/) {
        $offsetIdx--;
      }
      my $pos = $posAbsolute + ($tokens[$offsetIdx] - 1);
      $xmlPacket .= "<position>".$pos."</position>";

      # Add suggestions
      my @suggestions = ();
      if ($tokens[4]) {
        @suggestions = split(", ", $tokens[4]);
        for my $suggestion (@suggestions) {
          $xmlPacket .= "<suggest>$suggestion</suggest>";
        }
      }

      # End error element
      $xmlPacket .= "</error>";

    } elsif( $cmdReturn =~ /^$/ ) {
      # We have a blank line which indicates a line of text has been processed

      my $line = $lines[$lineNum];
      # Because of the UTF-8 Hack, do not add +1 to each line
      # - JHH 2005-07-09
      #$posAbsolute += (length($line) + 1);
      $posAbsolute += length($line);
      $lineNum++;
    }
  }
  close ASPELL;

} else {
  $xmlPacket .= "<exception>BBCorrector Server has encountered an error ($!)</exception>";
}

# Delete the temp file
unlink $tempFilename;

# End results XML packet
$xmlPacket .= "</spell-results>";

#print "check = $text2Check<p>\n";

# Return XML packet back to client
print "$xmlPacket\n";

[cbotd]

Ted, thanks so much. I got it to work right away. Very exciting.

I'm on t-mobile 7230 using BBWC no BES

I had to reboot by removing the battery twice to get the program to show in the menu

No username or password entered

Disabled MDS Proxy

THANKS AGAIN!

[tzarcone]

Originally Posted by cbotd

Ted, thanks so much. I got it to work right away. Very exciting.

I'm on t-mobile 7230 using BBWC no BES

I had to reboot by removing the battery twice to get the program to show in the menu

No username or password entered

Disabled MDS Proxy

THANKS AGAIN!

Thanks for everyones responses never thought so many people would be using it. But I think it is great to be able to help out and have so many great comments for running the public server. If there is anything else that needs hosting for the BB's I will be right there to put it on my server.

Now if I can only learn BB programing might have some applications also. But that is next already loaded the sdk on my workstation so will see what happens from there.

Ted

[XDS]

Someone on another forum mentioned A great Idea about running A "Zip Side loading extractore" using the NeVampire CGI plugin,"

(This sounds sooo cool)
Would you or some one be able to do something similiar ?!

This would help soooo much with files I can't get to till I get home.



I believe the site is.

http://www.NetVampire.com





Best Regards,

~X~


BTW I'm very new to the BB myself.

[gewfie]

Edit: Nevermind, he changed it to port 80!!! Nice work Ted! Working here on a BB 7290 w/ BES.

Is the link http://www.rebreathertech.com:81/cgi...ctorServer.cgi still working for anyone? I keep getting connection refused.

-gewfie

[tzarcone]

Port 80 is only working temporary, I was replacing drives in my main server and used the linux server to still host my webpages. I accidently deleted a foward for port 81 on my router but that is back in place so use port 81. I will be done the work on my other server most likely today and than port 80 will give you an error message.

[Soapm]

Corrector still won't show up in my menu with OS 4.0.2.32 but if I downgrade back to 4.0.0.198 it appears in the menus just fine. I get an uncaught exception error when my BB boots up with 4.0.2.32 and Corrector installed. I believe the error i'm getting has to do with corrector trying to merge itself into my menus. Below is a copy of the error from my debugging logs. Note this line, "ApplicationMenuItemRepository addMenuItem". Can someone make heads or tails of this?

Also, I get this exact same uncaught exception error from PD with OS 4.0.2.32 then BBReply doesn't appear in my menu. If I downgrade to 4.0.0.198, no error and BBReply appears in my menu just fine. Everything else regarding PD works just fine, just the uncaught exception error coupled with no BBReply on the menus and only when using OS 4.0.2.32. This is why I believe these errors have to do with the applications trying to merge themselves into my menus.

I was ok using 4.0.0.198 but it won't do the new MMS feature so I had to go back to OS 4.0.2.32.
===== Corrector error LOG =====
Name: Error
GUID: 9c3cd62e3320b498
Time: Jul 24, 2005 00:32:34
No detail message
net_rim_cldc-4
<unknown>
<unknown>
0x3923
net_rim_cldc-4
ApplicationRegistry
waitFor
0x37E3
net_rim_cldc-4
ApplicationRegistry
waitFor
0x374A
net_rim_bb_apps_framework
TaskCollectionHolder
getTaskCollection
0xAD5F
net_rim_pdap_todo
ToDoListImpl
<init>
0x1004
net_rim_cldc-1
Class
newInstance
0x1F35
net_rim_pdap
ToDoListFactory
createToDoList
0x9E7A
net_rim_pdap
PIMImpl
openPIMList
0x8131
net_rim_bbapi_menuitem
ApplicationMenuItemRepository$SdkProxyVerb
<clinit>
0x4F3
net_rim_bbapi_menuitem
ApplicationMenuItemRepository
addMenuItem
0x15A
bbcorrector
BBCorrector
<private>
0xE6
bbcorrector
BBCorrector
main
0x3F

===== PD Error LOG =====
Name: RuntimeException
GUID: 9c3cd62e3320b498
Time: Jul 23, 2005 23:39:56
ApplicationRegistry.waitFor(0x1c83a547b3934c66) timeout
net_rim_cldc-4
ApplicationRegistry
<private>
0x3911
net_rim_cldc-4
ApplicationRegistry
waitFor
0x37E3
net_rim_cldc-4
ApplicationRegistry
waitFor
0x374A
net_rim_bb_apps_framework
TaskCollectionHolder
getTaskCollection
0xAD5F
net_rim_pdap_todo
ToDoListImpl
<init>
0x1004
net_rim_cldc-1
Class
newInstance
0x1F35
net_rim_pdap
ToDoListFactory
createToDoList
0x9E7A
net_rim_pdap
PIMImpl
openPIMList
0x8131
net_rim_bbapi_menuitem
ApplicationMenuItemRepository$SdkProxyVerb
<clinit>
0x4F3
net_rim_bbapi_menuitem
ApplicationMenuItemRepository
addMenuItem
0x15A
PDUtils
PDMail
<init>
0x658
PDUtils
PDMail
main
0x6AD

[Soapm]

Well, long days effort made short, I was able to get rid of the uncaught exception error and get corrector and bbreply to appear in the menu by changing net_rim_pdap_todo.cod module with the one from version 4.0.0.198. I don't know why just I am having trouble with this but I redownloaded the OS, did many clean wipes and everything. Hopefully someone can talk this file over with RIM.

Now that I have corrector back in the menu I now have the corrector problem I had with 4.0.0.198. When corrector tries to connect to the server I get an error saying, "bbcorrector could not find a service book entry for IPPP". I have an entry called "IPPP for BIBS [IPPP]" in my service book so I don't know why corrector doesn't acknowledge it.

I tried checking and unchecking the Disable MDS proxy setting and when it is checked I get the inside/outside the firewall error due to IT Policy and when it is unchecked I get the IPPP error. Any suggestions to a guy whose had a very long day?

[cbotd]

The only thing I think you haven't looked at is the port number. It seems like you've covered everything.

[Soapm]

Thanks cbotd, I verified port is ok...

Has anyone gotten corrector to work when leaving the "Disable MDS Proxy" box unchecked? Or does everyone get "bbcorrector could not find a service book entry for IPPP"? Please respond either way...

I believe my problem lies here, when I check the box I get an IT policy error which I can't get around but I shouldn't have to check the box since that setting was to correct a Nextel problem. I have TMO so me and anyone not on Nextel should not have to check that box. This is why I need to know if anyone got corrector to work without checking that box? If not then the problem is with correctors coding, it doesn't properly find the IPPP service book.

[cbotd]

i have t-mobile service and have the following config:

http://www.rebreathertech.com:81/cgi...ctorServer.cgi

username: Left this blank
Password: Left this blank

Disable MDS Proxy: Checked

I'm on a 7230. Non BES OS 4.0.0.185

If I use the program without the cursor being on the first line of text, I get an error. Not the IPP error but a server error.

[pharma1]

7100t w/ 4.0.0.2.32 Handheld. Using BWC, no BES. http://www.rebreathertech.com:81/cg...ectorServer.cgi

username: Left this blank
Password: Left this blank

Disable MDS Proxy: Checked

Both Applications work.

[jibi]

Originally Posted by Soapm

Well, long days effort made short, I was able to get rid of the uncaught exception error and get corrector and bbreply to appear in the menu by changing net_rim_pdap_todo.cod module with the one from version 4.0.0.198. I don't know why just I am having trouble with this but I redownloaded the OS, did many clean wipes and everything. Hopefully someone can talk this file over with RIM.

Now that I have corrector back in the menu I now have the corrector problem I had with 4.0.0.198. When corrector tries to connect to the server I get an error saying, "bbcorrector could not find a service book entry for IPPP". I have an entry called "IPPP for BIBS [IPPP]" in my service book so I don't know why corrector doesn't acknowledge it.

I tried checking and unchecking the Disable MDS proxy setting and when it is checked I get the inside/outside the firewall error due to IT Policy and when it is unchecked I get the IPPP error. Any suggestions to a guy whose had a very long day?

It looks like your IT dept may have re-enabled MDS...

The only way to get rid of the inside/outside firewall error is to wipe your handheld again. After doing so, I'd load this up prior to getting yourself back onto the BES server via EA. If you do an EA, just delete the Desktop [BrowserConfig] and Desktop [IPPP] service books, then install the app, then regenerate a new encryption key via Desktop Manager or Security options.

Might work.

[Soapm]

I'll try that later Jibi but I have only 3 desktop service book entries, cical, alp & cmime. That's why i've concluded i'm not going to get around that inside/outside firewall error which only comes when the "Disable MDS Proxy" box is checked.

This made me turn my guns at why do I need to check that box if I'm not on Nextel? So far it seems everyone whose gotten corrector to work has to have that box checked which doesn't seem right. Why won't it recognize TMO IPPP service book entry?

[Soapm]

fyi...

No go on the wipe, I did several security wipes then java wipes but that inside/outside error won't go away. Thanks for the suggestion...

[Soapm]

Well, life is looking up and starting to blow instead of suck. After I read Scooby's post about TMO force enabling the firewall I decided to give them a call. After getting to second level in Canada, they explained that everyones firewall is forced enabled by RIM. Their logic, if one BB get's compromised, there is a potential to get to the BB network which would comp all including Gov etc...

He gave the same advice as Jibi so I wiped security settings, then OS I even tried wipe -a and wipe -f. I then removed DM and loaded it again with just BIS (no redirector). It worked! I can now use BBCorrector and any application I want.

I then reloaded DM for BES, put it back on the BES and it still works. If my IT guys are still reading this board, please accept my apologies and understand that I didn't know why my BB was malfunctioning. Because the error said IT Policy, it was natural to suppose the policy was causing the problem. The solution seemed extreme but I don't mind cuz IT WORKED!

===
New problem, after corrector finds a mispelled word, I select a word and do a suggestion which brings up a list of suggestions. When I select one of the suggestions, corrector not only replaces the mispelled word, it also removes some of the previous spaces/words etc...

ex. "My world iz looking up" corrects to "My worldis looking up..."

Anyone else have this problem?

[cbotd]

I haven't payed that close of attention to the space problem.

But man, someone outta send you a nice shiny quarter for all the work you've had to do.

I'll start montitoring my corrector corrections. Lord knows I have enough of them.

[tateu]

Originally Posted by Soapm

New problem, after corrector finds a mispelled word, I select a word and do a suggestion which brings up a list of suggestions. When I select one of the suggestions, corrector not only replaces the mispelled word, it also removes some of the previous spaces/words etc...

ex. "My world iz looking up" corrects to "My worldis looking up..."

Anyone else have this problem?

Is that an exact quote of the entire email that has the problem? If not, can you post the entire text from an email that has the problem? That would make it easy for some of us to try and troubleshoot. If I type that in and spell check it, it turns out fine.

I have found problems when spell checking email that contains certain symbols. You can read about it in early posts of this thead: http://www.blackberryforums.com/show...5885#post75885

The fix posted in that post doesn't work, though. A slightly better fix was posted here: http://www.blackberryforums.com/show...6020#post76020

or the php version, which I think is a better fix than the perl version: http://www.blackberryforums.com/show...5887#post75887

[Soapm]

exact quote...

I typed "this iz a test" it corrected to "thisis a test"

[Wolf6can]

Thank you tzarcone for hosting the server.