c# .NET HttpWebRequest slow - Expect 100 continue!

Another tech stuff, at least as I am not posting lately. So, we were sending HTTP requestso the server and against a previous (3rd party) version it was extremely slow. Any browsing would suggest to change buffering, async mode or turning off proxy, but none helped.

When we changed POST to GET, this problem dismissed!
When we rewrote using sockets, the problem dismissed, but could not take the risk of an own implementation on a production system.

Profiling showed the bottleneck is this line:

HttpWebResponse r = (HttpWebResponse)request.GetResponse();

the poll method. Finally the solution was

Expect 100 Continue

Never heard of? :) What a surprise, turn itt off and give it a try. For further explanation search the web for it.

System.Net.ServicePointManager.Expect100Continue = false;
or
request.ServicePoint.Expect100Continue = false;

Good luck.

VS Express Edition - Start external program solution

"As you've noticed, this functionality is not available in Express. This is one of the many places where the IDE has been simplified..." words of Luke Hoban on social.msdn.microsoft.com.

It's not simplyfication, it is just taking away basic functionality and getting money from users. The postbuild event idea is not bad, but will start the application every time and I am not sure about debugging, if you need to attach to a process or it is done automatically. Here comes a solution, you need to edit the .user file of your project: .vcproj.user for C++ or .csproj.user for C#, probably similar for VB.

For C++ find a Configuration and edit|add DebugSettings:

    <debugsettings
        Command="C:\Program Files\Adobe\FrameMaker7.1\FrameMaker.exe"
        WorkingDirectory=""
        CommandArguments=""
        Attach="false"
        DebuggerType="3"
        Remote="1"
        RemoteMachine=""
        RemoteCommand=""
        HttpUrl=""
        PDBPath=""
        SQLDebugging=""
        Environment=""
        EnvironmentMerge="true"
        DebuggerFlavor="0"
        MPIRunCommand=""
        MPIRunArguments=""
        MPIRunWorkingDirectory=""
        ApplicationCommand=""
        ApplicationArguments=""
        ShimCommand=""
        MPIAcceptMode=""
        MPIAcceptFilter=""
    />

C#:

<project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <propertygroup condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <startaction>Program</startaction>
    <startprogram>C:\Program Files\Adobe\FrameMaker7.1\FrameMaker.exe</startprogram>
    <startarguments/>
    <startworkingdirectory/>
  </propertygroup>
</project>

Good luck debugging your libraries with Express edition.

Instantchess reset free game limit

Ever wondered how they are doing (instantchess dot com)? Changing the nick soes not help. It works from the same network but different computers. So it is not filtered on IP. Cookies? Good one, but no! And that didn't let me sleep. Flash cookies!!! Very smart. Will not tell you how to clear those beauties but Google will show you the proper Macromedia page.

Oh, on the other hand any person familiar with web and flash, or just a well informed person knows things like this. Companies like to hide things in flash/cookies and many times they are also harnessing this feature. So just spreading common knowledge, think people, smart things make you stupid!

Workplace: they started to monitor internet browsing - solution

Hi, my employer or rather those beautiful administrators decided to monitor internet browsing, to reduce the network traffic. They were monitoring it even before, but not officially and were doing it only to add pages to blacklist. I'm so happy, now the 2Mb connection will be like Pegasus, flying when needed. Of course not.

But what to do when you are bored or asking for tasks is not the way to getting promoted :)
Fortunately we are intensively using virtual machines for different purposes, VMware. Set the network to bridged, that way you access your network directly and they will have no way to get to you, they will see only the ip and the name of the virtual machine (change it) :) Happy browsing anonymously and till admins will find a way (if exists, probably yes) to associate virtual machines to real ones :)

You know this is not ethical ;)

Uploading large files using HTTP PUT in .NET

Hi,

another tech blog to share my struggle with HTTP PUT. C# offers many ways to upload files, the simplest is using WebClient, however it failed with out of memory on large files.

During the testing I received many different problems

• System.NotSupportedException: The stream does not support concurrent IO read or write operations.
• System.OutOfMemoryException
• The server committed a protocol violation. Section=ResponseStatusLine
• And few more I forgot but will add later, maybe...

Out of memory:
HttpWebRequest request = new ...
request.SendChunked = true;

Concurrent IO operations:
HttpWebRequest request = new ...
request.AllowWriteStreamBuffering = false;

Protocol violation:
Create a web.config file near your exe with useUnsafeHeaderParsing=“true“, search web how to do that properly. But that was not acceptable in our case as we had both source codes, client and server. The server was not sending response or not a standard response with proper headers. So after uopload we removed the wait for a response:

HttpWebResponse response = (HttpWebResponse)request.GetResponse()

Hope I helped, good luck.

Annoying password policy - solution

Hi, for sure you have faced the welcome message of expired password. Then you thought you will change it quickly and continue. But, it needs uppercase, lowercase, numbers, signs, squirrel sounds as Dilbert said, zero is not allowed not to be missed by o and other specialties...

This happened to me last time and it made me quite angry. Finally when you have a new password, safety is still worthless because you are unable to remember your password and you write it down on your desk. Oups.

Keeping the same password is not allowed, you need to find out how many last passwords are remembered, change your password as many times and finally use your last one. It will take more time, but you can keep your old password. Is it right to do this? Hm, maybe not, but is it right to writing it down?

Flickr almost Pro account for free

Have you tryed Flickr? Probably yes and you are thinking about a Pro account as the free has limitations that after a short time are quite disturbing. Like only three galleries without embedding, only last 200 photos visible, ... And probably more, but only these are the main things pushing you to buy the pro account. But.

There is a way how to make it work without paying. Groups! Create as many groups as you want and make them public, invitation only. You can add your pictures into groups instead of galleries.

Good luck and enjoy while its not restricted ;)

C# - Variable xxx is already defined in this scope

WTF!?

after coming back from holidays I had no mood/time to write anything, so let's change it with a technical stuff. I was writing a C# code (C# is a programming language copyed from Java) and found a strange behaviour of variable scopes unique to this language. Create a for cycle, define a variable inside, define the same variable after the for cycle and an error is shown:

for (int i = 0; i < 10; i++)
{
int xxx = 1;
}

int xxx = 0; //ERROR!

Variable xxx is already defined in this scope.

What scope, that's a child scope and has nothing to do with this one. Arggg.

:D

Windows 7 with new NVIDIA driver results in oversaturated odd colors - solution

Here's how to set (probably) correct colors for nvidia on win7:

control panel ->appearance and personalization ->display ->change display settings->advanced settings->GeForce 9300M GS->start the NVIDIA Control Panel->adjust desktop color settings:

set digital vibrance to 0% from 79%

was it easy? what a smart person was it to set it as default and overriding even calibrated monitors. Congrats.

Use regular expression in your C++ program (vbscript and groups)

There's a great tutorial how to reuse VB6 regular expressions in C++ on sourceforge and here's how to use it in more depth. Unfortunately there are not many examples how to use it in C++ and figuring out how to access groups was not hard, just took a while. The key is to use the version 2 of all the classes, starting with IRegExp2Ptr. So just an example how to do it:

IRegExp2Ptr regExp;
regExp.CreateInstance(__uuidof(RegExp));
regExp->Pattern = _bstr_t("^(\\d+)-(..)-(.*)\\.44$");

if (regExp->Test("11-ab-33.44") == VBOOL_TRUE)
{
 IMatchCollection2Ptr  matches = regExp->Execute("11-ab-33.44");
 IMatch2Ptr match = matches->Item[0];
 ISubMatchesPtr sbm = match->GetSubMatches();
 _bstr_t val = _bstr_t(sbm->Item[0]); //11
 _bstr_t val = _bstr_t(sbm->Item[1]); //ab
 _bstr_t val = _bstr_t(sbm->Item[2]); //33
}

MSXML append xml fragment as child

A C/C++ method used to append XML text to an MSXML node/element as child. The appendChild function can append only node/element objects. Sometimes it's easier or more convenient to create the XML with text concatenation than by building it using MSXML. And the code:

void appendXmlText(MSXML2::IXMLDOMElementPtr elem, _bstr_t xmlText)
{
 //only one root elem is allowed!
 _bstr_t xmlFragment = "<fragment>" + xmlText + "</fragment>";

 MSXML2::IXMLDOMDocument2Ptr _doc;
 TESTHR(_doc.CreateInstance(__uuidof(MSXML2::DOMDocument40)));
 _doc->validateOnParse = false;
 _doc->resolveExternals = false;
  
 _doc->loadXML(xmlFragment);
 MSXML2::IXMLDOMElementPtr root = _doc->documentElement;
 MSXML2::IXMLDOMNodeListPtr childs = root->childNodes;
 long childCount = root->childNodes->length;
 for (long i = 0; i < childCount; ++i)
 {
  MSXML2::IXMLDOMNodePtr childNode = root->childNodes->item[i];
  elem->appendChild(childNode->cloneNode(VBOOL_TRUE));
  //VBOOL_TRUE is -1
 }
}

C/C++ CreateProcess command line param problem

Back to some tech stuff, a simple task yesterday made me much headaches. I needed to run pdftk.exe from our program. I figured out the params and all worked as a dream. Let's implement it:

CreateProcess("pdftk.exe", "a.pdf b.pdf cat output c.pdf dont_ask", ...)

And it did not work. I failed to get any error, any output. Then I tried with cmd:

CreateProcess("cmd.exe", "/C pdftk.exe a.pdf b.pdf cat output c.pdf dont_ask", ...)

And it worked, but why? And I don't want the cmd.exe to be involved. I saw an example where program name was null:

CreateProcess(NULL, "pdftk.exe a.pdf b.pdf cat output c.pdf dont_ask", ...)

And it worked, ok, we got rid of cmd.exe.
WTF? The program name is null and the params, wait, command line!

The flexibility of the CreateProcess() function (and a possible point of confusion) arises when you pass a valid string pointer to both the ApplicationName and CommandLine parameters. This allows you to specify the application to be executed as well as the complete command line that is passed to the application. One might assume that the command line passed to the created application is a composite of the ApplicationName and CommandLine parameters, but this is not the case. As a result, a process created by CreateProcess can receive a value other than its .exe name as its "argv[0]" parameter. The following is an example of a call to CreateProcess that produces this "abnormal" behavior:

   CreateProcess( "c:\\MyApp.exe", "Param1 Param2 Param3", ...)
 

MyApp's arguments will be as follow:

  argv[0] == "Param1"
  argv[1] == "Param2"
  argv[2] == "Param3"

Read more about it here. Hope it will save you some time :)

Saveing images from Lightbox

Another hack how to save images from web pages that try to implement some protection,
this time Lightbox.

• Disable javascript support in your browser, after clicking the image is displayed.

After experimenting, you'll find many other ways how to save images. You can also save images from Lightbox list too (slide shows).

Canon EOS burst buffer

I was shooting high ISO pictures in burst mode (1/200 - F2) and the 40d dropped the burst buffer for RAW to 6 pics or so and I was not sure why. This was very limiting and I could not make it work. The answer is easy: RTFM (page 156)! I did it before many times but who remembers all those settings :)

Turned on/automatic high ISO noise reduction significantly reduces the RAW buffer! Turn it off and do it in computer.

Creating PDF for free from any program with printer support

This is a short manual how to replace any commercial and/or closed source PDF distilling product. What you need:

• Adobe Universal Postscript Driver
• PPD Printer Driver (get one from prev link or create one for yourself)
• Ghostscript

Why would you need that?
There were many reasons we decided to replace Adobe Distiller and Acrobat:

• Programmatic control of these programs!
• Incompatibility of different versions of Framemaker/Distiller/Acrobat!!!
• Adobe bugs!!!
  
• Price!
• Page size limits*
  
• Different page sizes in one document*
• Page inserting/removing bugs
• Language/Font problems*

How to print?
First, install the postscript driver (PPD). Change the PPD file for your needs, we changed limits for maximum paper size (maps). The printer should print into file. When printer is installed, you can print from any program with printing support, just choose this printer and the output will be saved as postscript (ps). Converting postscript to PDF is easy with Ghostscript:

gswin32c.exe -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sOutputFile=output.pdf input.pdf

We suggest to set embedding of fonts into postscript/pdf as ghostscript fonts can be buggy.
The same process can be done very easily even programmatically, just use google/msdn how to set printer to use.

Ghostscript Font alignment problems

We use Ghostscript for creating PDF documents, it is a much cheaper solution than buying special printer drivers, Adobe Distiller or any other. We changed Distiller for Ghostscript because of many reasons, but first of all bugs and control problems from other soft. But this is a different story.

Ghostscript uses free URW fonts for PDF creating and as we found out, theese fonts are buggy! As there won't be soon any update, we had to find another solution. We convert postscript (ps) files to PDF, so the solution was to embed the used font in the postscript instead of using the built in fonts.

In windows: printer preferences -> TrueType font: Download as SoftFont

From C/C++ set PRINTER_INFO_2 pDevMode->dmTTOption = DMTT_DOWNLOAD;

Good luck

Adobe Framemaker - Czech and Slovak language patch

Uspesne sa mi podarilo vytvorit patch pre Adobe Framemaker, ktory prida podporu cestiny a slovenciny. Je to univerzalny patcher, co znamena ze funguje so vsetkymi verziami Framemaker od verzie 5.* az po 7.*. Osmicka uz ma podporu unicode, tam je to nepotrebne. Skorsie verzie som nemal moznost odskusat, ale mohlo by to ist. Ak to niekoho zaujima, nech sa ozve (vlavo je contact me). Alebo mi nechaj odkaz, keby to neslo, niekedy to vypadava.

Case sensitive table names in oracle - WTF

We imported an M$Sql database to Oracle and the table names were case sensitive, all like tWHATEVER, tUSERS. Our program was not working because of bad SQL commands. After renaming a table (all uppercase) it becomes case insensitive.

Solution 1: rename all tables by hand using a tool (like TOAD)
Solution 2: write a script that does it for you, renaming by hand hundreds or thousands of tables is not an option. Here's the script, it's not error prone but works.

Known bugs: if table name is already case insensitive, it fails on renaming.

DECLARE
  string varchar2(200);
BEGIN
for tabs in (SELECT table_name, upper(table_name) as uppername FROM user_tables)
loop
  string := 'alter table ' || tabs.table_name || ' rename to ' || tabs.uppername;
  execute immediate string;
end loop;
END;

Lightroom 2 - edit more pictures in Photoshop Disabled

So, I began to use Lightroom 2 and after a time a really like it despite it needs a strong machine. I use it in combination with Photoshop 10.0. So when you select more pictures and want them to edit in photoshop, in context menu the options for editing all in layers, panorama, hdr are disabled.

Solution: LR2 requires photoshop CS3 10.0.1, you need to update! That's it. Also there's a LR2.1 patch coming, already in test phase that should fix many bugs.

Annoying spaceball.gif on flickr

As last week I created my flickr stream I also was exploring other ones. I saved some interesting pics as a future inspiration for my photos and found out that when you click right and want to save the pic, it only saves some weird spaceball.gif. WTF?!
Hm, this gif file is transparent image over the real image so you can't click on it and save. There are many ways to do get behind this, through different plug-ins, exploring cache, ... I present you two of them, solution for firefox users (but reproducable in other browsers):

• The easy one that needs no plugin:
  1. click right on the page, not the picture
     
  2. click view page info
  3. click media tab
  4. find the image and click save
  5. viola! protection gone ;)
• Get rid of spaceball.gif for ever
  1. you need a plug-in that blocks advertisements, I used ad-block plus
  2. create a filter for spaceball.gif (e.g. http://*/spaceball.gif)
  3. viola! no more spaceball, it's gone for ever

Hope it helped, good luck

Other howtos: Save images from lightbox