Problem:
Enter the URL to a small image file (or any file), retrieve the file and encode the data as base64, then decode the base64 to get the original files data.
When the page below is executed, the file is retrieved and encoded, however when the decode_base64() subroutine is called, the system displays the following exception:
System.FormatException: Invalid character in a Base-64 string.
at System.Convert.FromBase64String(String s)
at ASP.test_base64_aspx.decode_base64()
Obviously, there is an invalid character in the Base-64 string -- the strange thing is if I increase "k_buf_size" from 4096 to something like 32000 (in the encode_base64 subroutine) and execute the page everything works just fine!!
So my question is, why will this code NOT work with smaller buffer sizes?? Hopefully, I have made a simple programming error that someone will point out as I have stared at this for too long ...
ANY help would be appreciated!
<%@dotnet.itags.org. Page Language="VB" Explicit="true" Strict="true" Trace="true" %>
<%@dotnet.itags.org. Import Namespace="System.IO" %>
<%@dotnet.itags.org. Import Namespace="System.Data" %>
<%@dotnet.itags.org. Import Namespace="System.Net" %>
<script language="VB" runat="server">
' The base64 encoded data ...
private m_encoded_data as StringBuilder
' The size of the file that was encoded ...
private m_file_size as Integer
'----------------
'
private sub do_test( Sender as Object, e as EventArgs )
' Init ...
m_encoded_data = new StringBuilder()
m_file_size = 0
' Attempt to encode the input file ...
encode_base64()
' If a file was encoded, attempt to decode ...
if ( m_encoded_data.length > 0 ) then
decode_base64()
end if
end sub
'----------------
'
private sub encode_base64()
const k_buf_size as Integer = 4096
dim bytes_read as Integer
dim n as Integer
dim err_msg as String
dim s as String
dim url as String
dim sr as Stream
dim wc as WebClient
Trace.Warn( ">", "==============" )
Trace.Warn( ">", "encode_base64:" )
' Get URL ...
url = fld_url.text.Trim()
if ( url.length = 0 )
exit sub
end if
' Attempt to get the data file and encode as base 64 ...
try
wc = Nothing
sr = Nothing
m_file_size = 0
' Create space to hold data ...
dim data_array( k_buf_size ) as Byte
' Create web client ...
wc = new WebClient()
' Create stream from URL ...
sr = wc.OpenRead( url )
' Loop to encode data stream ...
' Read a chunk from the data steam into array ...
bytes_read = sr.Read( data_array, 0, k_buf_size )
do while ( bytes_read > 0 )
Trace.Warn( ">", "bytes_read = " & bytes_read.ToString() )
m_file_size = m_file_size + bytes_read
' Convert data chunk to base 64 string. Remove any padding characters (=)
' that may have been added ...
s = System.Convert.ToBase64String( data_array, 0, bytes_read )
n = s.IndexOf( "=" )
if ( n > 0 ) then
m_encoded_data.Append( s.Substring( 0, n ) )
else
m_encoded_data.Append( s )
end if
bytes_read = sr.Read( data_array, 0, k_buf_size )
loop
' Encoded output length must be a multiple of 4 bytes, so pad if necessary ...
n = m_encoded_data.length Mod 4
if ( n <> 0 ) then
m_encoded_data.Append( new String( "="C, 4 - n ) )
end if
' Clean up ...
data_array = Nothing
Trace.Warn( ">", "File size = " & m_file_size.ToString() )
Trace.Warn( ">", "Encoded data length = " & m_encoded_data.length.ToString() )
catch ex as Exception
Trace.Warn( ">", ex.ToString() )
finally
if ( not sr is Nothing ) then
sr.Close()
sr = Nothing
end if
if ( not wc is Nothing ) then
wc.Dispose()
wc = Nothing
end if
end try
end sub
'----------------
'
private sub decode_base64()
dim data_array() as Byte
Trace.Warn( ">", "==============" )
Trace.Warn( ">", "decode_base64:" )
' Attempt to decode from base64 ...
try
data_array = Convert.FromBase64String( m_encoded_data.ToString() )
Trace.Warn( ">", "data_array.length = " & data_array.length.ToString() )
if ( data_array.length = m_file_size ) then
Trace.Warn( ">", "DECODE OK" )
else
Trace.Warn( ">", "DECODE FAILED!" )
end if
catch ex as Exception
Trace.Warn( ">", ex.ToString() )
end try
end sub
</script>
<html>
<head>
<title>Attempt to encode/decode using Base 64</title>
</head>
<body>
<blockquote>
<form runat="server">
Enter the URL to a file, preferably a small image JPG or GIF file.
<hr noshade size="1">
<table width="90%" border="0" cellpadding="2" cellspacing="1">
<tr>
<td width="20%" align="right" valign="top" >
URL:
</td>
<td width="80%" align="left" valign="top" >
<asp:TextBox id="fld_url" text="" size="100" maxlength="200" runat="server" />
</td>
</tr>
</table>
<hr noshade size="1">
<table width="90%" border="0" cellpadding="2" cellspacing="1">
<tr>
<td width="20%"> </td>
<td width="80%" valign="top">
<asp:Button OnClick="do_test" text="Do Base64 Test" runat="server" />
</td>
</tr>
</table>
</form>
</blockquote>
</body>
</html>
Ok - here is one solution (may not be the best, but it works)...
The problem with base 64 encoding is that one has to be careful when reading data in "chunks" and then encoding each chunk (as base 64) and then appending each chunk to form the final encoded string -- this is what I was doing initially. The code would work whenever I changed the buffer size to anything larger than the file being read, ie: whenever the entire file was read in one chunk. In any case, the solution is shown below for anyone that comes across this problem. Please note that the solution below limits the file sizes that can be encoded to 2 megabytes.
<%@. Page Language="VB" Explicit="true" Strict="true" Trace="true" %>
<%@. Import Namespace="System.IO" %>
<%@. Import Namespace="System.Data" %>
<%@. Import Namespace="System.Net" %>
<%@. Import Namespace="rentsys" %>
<script language="VB" runat="server">
' The base64 encoded data ...
private m_encoded_data as String
' The size of the file that was encoded ...
private m_file_size as Long
'----------------
'
private sub do_test( Sender as Object, e as EventArgs )
' Init ...
m_encoded_data = String.Empty
m_file_size = 0
' Attempt to encode the input file ...
encode_base64()
' If a file was encoded, attempt to decode ...
if ( m_encoded_data.length > 0 ) then
decode_base64()
end if
end sub
'----------------
'
private sub encode_base64()
' Maximum file size is 2 MB ...
const k_max_file_size as Long = 2000000
' Buffer used to read file in chunks ...
const k_data_buf_size as Integer = 4096
dim data_buf( k_data_buf_size ) as Byte
dim bytes_read as Integer
dim err_msg as String
dim s as String
dim url as String
dim ms as MemoryStream
dim sr as Stream
dim wc as WebClient
Trace.Warn( ">", "==============" )
Trace.Warn( ">", "encode_base64:" )
' Get URL ...
url = fld_url.text.Trim()
if ( url.length = 0 )
exit sub
end if
' Get the size of the remote file, unfortunately, the only way
' to reliably get this is to read the entire file. We could issue
' an HTTP HEAD request to get the content length, but some servers
' may not return file headers ...
try
wc = Nothing
sr = Nothing
m_file_size = 0
' Create web client ...
wc = new WebClient()
' Create stream from URL ...
sr = wc.OpenRead( url )
' Loop to determine file size ...
bytes_read = sr.Read( data_buf, 0, k_data_buf_size )
do while ( bytes_read > 0 )
m_file_size = m_file_size + bytes_read
bytes_read = sr.Read( data_buf, 0, k_data_buf_size )
loop
catch ex as Exception
Trace.Warn( ">", ex.ToString() )
finally
if ( not sr is Nothing ) then
sr.Close()
sr = Nothing
end if
if ( not wc is Nothing ) then
wc.Dispose()
wc = Nothing
end if
end try
' Check if we can handle the file size ...
Trace.Warn( ">", "File size = " & m_file_size.ToString() )
if ( m_file_size > k_max_file_size ) then
Trace.Warn( ">", "File size exceeds maximum allowed" )
exit sub
end if
' Attempt to read the file contents and encode as base 64 ...
try
wc = Nothing
sr = Nothing
ms = Nothing
' Create web client ...
wc = new WebClient()
' Create stream from URL ...
sr = wc.OpenRead( url )
' Create memory stream to hold file contents ...
ms = new MemoryStream()
' Loop to read data into memory stream. Note that we
' CANNOT convert each chunk read into base 64 and append
' to a string as this would lead to an invalid base 64
' string due to the nature of the algorithm and padding
' characters that would be present in the final string ...
bytes_read = sr.Read( data_buf, 0, k_data_buf_size )
do while ( bytes_read > 0 )
ms.Write( data_buf, 0, bytes_read )
bytes_read = sr.Read( data_buf, 0, k_data_buf_size )
loop
' Convert the data to base 64 ...
m_encoded_data = Convert.ToBase64String( ms.ToArray() )
catch ex as Exception
Trace.Warn( ">", ex.ToString() )
finally
if ( not ms is Nothing ) then
ms.Close()
ms = Nothing
end if
if ( not sr is Nothing ) then
sr.Close()
sr = Nothing
end if
if ( not wc is Nothing ) then
wc.Dispose()
wc = Nothing
end if
end try
end sub
'----------------
'
private sub decode_base64()
dim data_array() as Byte
Trace.Warn( ">", "==============" )
Trace.Warn( ">", "decode_base64:" )
' Attempt to decode from base64 ...
try
data_array = Convert.FromBase64String( m_encoded_data )
Trace.Warn( ">", "data_array.length = " & data_array.length.ToString() )
if ( data_array.length = m_file_size ) then
Trace.Warn( ">", "DECODE OK" )
else
Trace.Warn( ">", "DECODE FAILED!" )
end if
catch ex as Exception
Trace.Warn( ">", ex.ToString() )
end try
end sub
</script>
<html>
<head>
<title>Attempt to encode/decode using Base 64</title>
</head>
<body>
<blockquote>
<form runat="server">
Enter the URL to a file, preferably a small image JPG or GIF file.
<hr noshade size="1">
<table width="90%" border="0" cellpadding="2" cellspacing="1">
<tr>
<td width="20%" align="right" valign="top" class="form_label_1">
URL:
</td>
<td width="80%" align="left" valign="top" class="form_field_1">
<asp:TextBox id="fld_url" text="" size="100" maxlength="200" runat="server" />
</td>
</tr>
</table>
<hr noshade size="1">
<table width="90%" border="0" cellpadding="2" cellspacing="1">
<tr>
<td width="20%"> </td>
<td width="80%" valign="top">
<asp:Button OnClick="do_test" text="Do Base64 Test" runat="server" />
</td>
</tr>
</table>
</form>
</blockquote>
</body>
</html>
