My application can't find any of its files when it is started from the shortcut created by Setup. It works fine when I double-click the application's EXE in Explorer.
Your application is most likely not specifying pathnames on the files it is trying to open, so it is expecting to find them in the current directory. Inno Setup by default does not set the "Start In" field on shortcuts its creates; this causes Windows to pick a directory itself, which usually won't be the directory containing your application.

In virtually all cases, this is something that should be corrected at the application level. Properly designed GUI applications should not expect to be started from a particular directory; they should always specify full pathnames on files they open. In Delphi or C++Builder, for example, it's possible to get the full pathname of the directory containing the application EXE by calling: ExtractFilePath(ParamStr(0)). To get the full path of a file named "File.txt" in the application directory, use: ExtractFilePath(ParamStr(0)) + 'File.txt'.

If for some reason you cannot fix this at the application level, you can tell Inno Setup to set the "Start In" field by adding "WorkingDir: {app}" to your [Icons] entries.

C:\Programme
        \MeinProgramm
            \MeinProgramm (enthaelt die Perl-Skripte und -Module)
            \windows (enthaelt die Perl-Distribution)