Skip to content

Instantly share code, notes, and snippets.

@Ovid
Last active March 21, 2022 06:51
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Ovid/5205534f7dcc52e4d931aaff301b39aa to your computer and use it in GitHub Desktop.
Save Ovid/5205534f7dcc52e4d931aaff301b39aa to your computer and use it in GitHub Desktop.
Exceptions in Perl?

Preface

This is something that likely cannot be made into an RFC for the Perl language at this time because implementation would be greatly simplified when the Corinna object model is in core. For example, a base class for what is discussed might look like the following:

# Exception is a poor name for warnings, so a better name is warranted
class Exception :version(v0.1.0) {
    # $message and $description might be from a messaging role
    field $message     :reader :param;
    field $description :reader :param { "" };
    
    # must not be lazy because stack frames might not be available later
    field $stack_trace :reader { $self->build_stack_trace };
    
    method build_stack_trace :private () {
        # generate stacktrace
    }
    
    method throw () {
        die $self->to_string;
    }
    
    method to_string () {
        return join "\n", $message, $description, $stack_trace;
    }
}

Exceptions

Here's a common problem in Perl:

try {
    $object->some_method;
}
catch ($error) {
    if ( $error =~ /File doesn't exist/ ) {
        ...
    }
    else {
        croak($error);
    }
}

And some programmer sees the File doesn't exist string in the code and changes to the more standard File not found, but misses error handling that's based on string matching.

But exceptions make this safer:

try {
    $object->some_method;
}
catch ($exception) {
    if ( $exception isa Exception::FileNotFound ) {
        ...
    }
    else {
        $exception->throw;
    }
}

With that, you're testing the exception class, something far safer. In fact, at this point, you can localize exception strings. In French, you might get « Fichier introuvable » which, for a French speaker, might be much easier to understand. This wouldn't change the class of the exception, so it's more maintainable.

Exception objects, of course, could have a message, a more verbose description, a stacktrace, and so on.

Exception Hierarchy

What would an exception hierarchy look like in Perl, though? Would it be a tree or a graph (inheritance versus roles). I'm assuming Perl wouldn't handle "checked" exceptions (exceptions that perl requires be trapped at compilation time), but there would still need to be some organization.

For example, in Java, we have both Error and Exception, separate related classes, each of which inherits from Throwable. An Error is thrown when there's an unrecoverable error in a program, such as an AbtractMethodError where a subclass has failed to override an abstract method in a parent class. Even though code might trap an Error (for logging or sending email, for example), it should still probably rethrow that error.

An Exception, on the other hand, is an error that might be recoverable. A FileNotFound exception might cause the program halt, or it might prompt the user to manually enter values that should be in a config file.

However, we can't blindly port over an exception system from other languages. This is Perl, after all. Just reading through the Java error/exception hierarchy makes it clear that much of it doesn't apply to a dynamic language.

But what about this case? (A common issue with tests)

my $warning = capture_stderr {
    $object->some_method;
};
like $warning, qr/$some_warning/, 'We have our expected warning';

And someone changes the warning message and tests fail (guess what I've had to work on yesterday?) What if warnings were objects?

my $warning = capture_stderr {
    $object->some_method;
};
ok $warning isa Warning::Category, 'We have our expected warning';

So actually, we might want a generic messaging mechanism (localizable, I should think) that can be broadly applicable to many problem spaces, or we might want warning objects which are not throwable, though you could probably promote them to an exception and throw them.

And that got me thinking.

Exception Levels

If you use logging software in Perl, you often find "log levels". In the excellent Log::Log4Perl module, here are the log levels:

$logger->trace("...");  # Log a trace message
$logger->debug("...");  # Log a debug message
$logger->info("...");   # Log a info message
$logger->warn("...");   # Log a warn message
$logger->error("...");  # Log a error message
$logger->fatal("...");  # Log a fatal message

I couldn't help but think that "exception levels" might be a better way to think about an exception hierarchy (at which point, the word "exception" doesn't seem appropriate), but now things get interesting. Imagine this:

use exceptions 'warn';

sub first () {
    info "This is a no-op due to the exception level";
}

sub second () {
    warn "This is trappable, but untrapped, prints this message";
}

sub third () {
    error "This is trappable, but untrapped, prints this message, a stacktrace, and dies";

sub fourth () {
    fatal "This is probably trappable (?), but untrapped, prints this message, a stacktrace, and dies";
}

No, I am not suggesting the above syntax (too many clashes with existing Perl). Further, I can see pitfalls with this (easy to swallow warnings), but I wanted to start a discussion about a conceptual framework of different types of informational messages that the program might emit.

All of the above might consume a "Message" role to make it easier to localize messages for various spoken languages. Said role might even be more generally useful outside of exceptions, though this might be unworkable due to the need to maintain translation files. Careful thought might be needed here.

Corinna

Many things for the above would be easier to implement when the Corinna object system gets into the Perl core (work on this is expected to begin after the next major release).

Conclusion

Currently, Perl has no native way of generating exception objects and the CPAN has tons of competing different implementations. Proper exceptions can offer a clean standard, make code more robust, but the dynamic nature of Perl suggests to me that we might want to think about exceptions in a new way.

@duncand
Copy link

duncand commented Mar 2, 2022

There's nothing particularly special about "exception objects". They are just objects you happen to use as exceptions. Anything can be used as an exception. An exception is just a control flow and message passing mechanism.

@duncand
Copy link

duncand commented Mar 2, 2022

This may not have been intentional, but this post comes across like a number of other proposals or implementations I have seen that try to look at exception types like they are their own isolated thing rather than taking a holistic approach of a wider type system.

While I can understand breaking down the problem, it seems very wrong for Perl to have a canonical set of built-in classes specific to exceptions without also having a canonical set of built-in classes for things that are not exceptions.

I feel that BEFORE we think of having an official set of exception classes, we should have official strong type support and canonical classes for things people use for regular data all the time.

However, something I CAN get behind early is having a SINGLE generic built-in Exception class with a plain scalar string property or such which is the type of exception, and other properties for other useful metadata. So NO hierarchy of distinct classes, use a string to differentiate instead.

That would be a lot more Perlish. And this one class can still have its objects stringily like the old textual exceptions built-in.

@Ovid
Copy link
Author

Ovid commented Mar 2, 2022

@duncand Exceptions should be used for exceptions, not flow control. Early Java developers learned this the hard way. Generating a stacktrace can be very expensive as you walk up the stack frames. Using exceptions all over the place for flow control causes significant performance issues.

Instead, a good rule of thumb for exception design is that, assuming nothing "unusual" happens in the program, if all exceptions were removed, the program should still be correct.

It's similar to the issues with AOP (aspect-oriented programming): if you remove aspects and the program still runs correctly, that's good. If your program depends on AOP to function correctly, it can be a nightmare when you miss a pointcut. A friend learned this the hard way when debugging a threaded Java program that used method name matching to decide which methods needs to be locked for threads. This led to a bug where a developer renamed a method and failed to notice that the pointcut no longer matched.

@Ovid
Copy link
Author

Ovid commented Mar 2, 2022

@duncand wrote:

However, something I CAN get behind early is having a SINGLE generic built-in Exception class with a plain scalar string property or such which is the type of exception, and other properties for other useful metadata. So NO hierarchy of distinct classes, use a string to differentiate instead.

I think that might be appropriate, but imagine if we had this:

use exception::level 'warn';

try {
    open my $fh, '<', $filename;
}
catch ($exception) {
    if ( $exception isa `FileNotFound` ) {
        # prompt user for location of file
    }
    elsif ( $exception isa `FileNotReadable` ) {
        # take some more appropriate action
    }
    else {
        $exception->throw;
    }
}

Something like an 'autoexception' mechanism could be very useful, and for any area where Perl already has "errors" (system, open, etc.), having a native exception type for them seems to make sense and we can have a standardized way of dealing with them.

A generic exception mechanism would be useful for custom exception issues. For standard issues, built-in exceptions which don't reflect what Perl currently does means re-implementing them all over the place. If exceptions are merely behavior without semantics, we get back to the core problem:

try {
    $object->some_method;
}
catch ($exception) {
    if ( $exception isa 'Exception' && $exception->message =~ /File doesn't exist/ ) {
            ...
    }
    else {
        croak($error);
    }
}

I'm tired of going into a client to find yet another way of handling common errors.

  • Sometimes they die
  • Sometimes they croak
  • Sometimes they confess
  • Sometimes they're ignored
  • Sometimes they throw an exception object
  • Sometimes that exception object doesn't have a stack trace
  • Exception message strings are ad hoc and unreliable (behavior without semantics)
  • ... and so on

And I would still love to have something analogous for warnings, information messages, debug messages, tracing messages, and so on. Those, presumably, would not be thrown (or being thrown might just dump the description to STDERR).

To be fair, all of this would be a huge undertaking. For a first pass, a generic mechanism is appropriate and later, we can fine-tune exceptions by returning subclasses of the generic mechanism, giving developer even better control over what they're doing.

@duncand
Copy link

duncand commented Mar 2, 2022

@duncand Exceptions should be used for exceptions, not flow control. Early Java developers learned this the hard way. Generating a stacktrace can be very expensive as you walk up the stack frames. Using exceptions all over the place for flow control causes significant performance issues.

@Ovid When I say "control flow" I mean that in the generic sense that exceptions are a method for abnormal exit from a block or several.

@duncand
Copy link

duncand commented Mar 2, 2022

@Ovid The idea of my proposal was to have a single Exception class with a main string property which is tested with equality and that there is NOT regular expression matching. This string would exist instead of having a class hierarchy. So kind of like this:

try {
    open my $fh, '<', $filename;
}
catch ($exception) {
    if ( $exception->kind eq 'FileNotFound' ) {
        # prompt user for location of file
    }
    elsif ( $exception->kind eq 'FileNotReadable' ) {
        # take some more appropriate action
    }
    else {
        $exception->throw;
    }
}

This is almost identical to your example but there are no subclasses.

@duncand
Copy link

duncand commented Mar 2, 2022

When considering an actual type hierarchy for exceptions, I believe the concept should be generalized, and perhaps not use the name "exception". The generalization of the concept, which I would call Excuse, or Raku calls Failure or some such, is you have objects which represent that some operation had an abnormal result, and so this is expressed as a value of some type that is a different type than the operation normally results in. One should be able to treat these values as normal data including that the NORMAL way to express them is via ordinary returned values. For example, the division operator normally returns a number but that if you tried to divide by zero it returns a Div_By_Zero object instead. You could ALTERNATELY throw these values using an exception mechanism, which aborts the block early etc, but you DON'T HAVE TO. The key issue is that expressing abnormal result is decoupled from whether an abnormal return happens or not. So we have at least 2 orthogonal things to design here. As to the matter of whether returning special values can easily be ignored as per traditional Perl file operators that don't die, well having my proposal coupled with a stronger type system that throws an exception if you try to assign the result to something that only accepts a normal result type and not also the excuse type, you'd get an exception at that point like when trying to assign a string to a number under strong typing.

@duncand
Copy link

duncand commented Mar 2, 2022

So what I propose is a nesting system. You have ONE generic Exception class, which has all the special stuff like the ability to throw/catch it and having all the stack trace stuff or whatever. And that Exception class has a $kind property whose value is an Excuse/Failure object, and it is only the latter that has a whole type hierarchy that has the interesting specific details of the situation like FileNotFound vs FileNotReadable vs DivByZero etc. And wrapping an Excuse/Failure in an Exception is OPTIONAL, depending whether or not you want that stack trace stuff or not, and an Excuse/Failure can be returned as a result rather than being thrown, which is necessary for many kinds of parallel processing or functional programming styles.

@Ovid
Copy link
Author

Ovid commented Mar 2, 2022

@duncand Cheers for the clarification regarding flow control. Will read the rest later.

@Ovid
Copy link
Author

Ovid commented Mar 2, 2022

@duncand wrote:

try {
    open my $fh, '<', $filename;
}
catch ($exception) {
    if ( $exception->kind eq 'FileNotFound' ) {
        # prompt user for location of file
    }
    elsif ( $exception->kind eq 'FileNotReadable' ) {
        # take some more appropriate action
    }
    else {
        $exception->throw;
    }
}

This is almost identical to your example but there are no subclasses.

Here we get some semantics, but not enough. If I want to have special handling for all file type exceptions and different handling other types of exceptions, your suggestion requires something like if ( $excetion->kind =~ /^File/ ) { ... } This also requires that someone else writing a custom file-based exception respects the naming convention and hoping that someone else writing, say, a FileSystem exception realize that you're string matching and maybe the naming is wrong. (You might argue that FileSystem is a dumb name for an exception, but I could easily come up with plenty of other examples, so don't get stuck on "files" versus "filesystem").

But if we inherit from a generic FileException class, it's safe because we've built in the proper semantics:

try {
    open my $fh, '<', $filename;
}
catch ($exception) {
    if ( $exception isa 'FileException' ) {
        # do something
    }
    elsif ( $exception isa  'ArithmeticException' ) {
        # do something else
    }
    else {
        $exception->throw;
    }
}

Or we want to handle "file not found" but do something different for all other file-based exceptions: trivial, safe, and no string matching.

In other words, by designing a proper hierarchy and using that to encode more specialized intent in subclasses (which was the original intent of OOP), we get a system that's provides the proper type of information to programmers. Please focus on semantics, not just behavior.

@duncand
Copy link

duncand commented Mar 2, 2022

@Ovid Focusing on semantics is the point of my later proposal which overrided the one you replied to. My latest version uses a class hierarchy but that hierarchy is SPECIFICALLY ABOUT ENCODING SEMANTICS and it is NOT specific to exceptions. And I want to repeat that all versions of my proposal meant NEVER using regular expressions or looking at substrings.

@duncand
Copy link

duncand commented Mar 2, 2022

On a tangent, I believe ALL types should include semantics, with LIMITED structural exceptions. So for example, with NORMAL user data types, users should NOT be using plain generic integers or strings, but should instead at a minimum be using wrapper types that provide meaning. So for example, they should have CompanyID and ProductID classes rather than integers or strings, and PersonName classes rather than strings, and Currency or Quantity etc rather than numbers. A typed exception hierarchy fits better when the normal data does this too, and not just the stuff indicating that normal data is missing.

@Ovid
Copy link
Author

Ovid commented Mar 2, 2022

@duncand Now I get what you're saying. I didn't think deeply enough about your later proposal. I find that interesting. How can that be trivially composed with the idea of "exceptions as log levels" (or is the latter idea too weird?)

@duncand
Copy link

duncand commented Mar 3, 2022

@Ovid I consider logging and log levels to be another dimension orthogonal to either abnormal result values or exceptions. In practice logging is frequently done in contexts not involving exceptions, though exceptions are often logged.

In any event, I consider the abnormal result values to be the core concept of interest to get right and that exceptions or logging are something that can be associated with it but are things I don't think about as much.

I also want to say that as I've conceived it the Excuse concept is even more broad, and serves to also handle all of the use cases of SQL nulls, and of IEEE float NaNs, and to express user input validation failures such as Invalid_Phone_Number or Input_Field_Wrong etc.

Generally speaking, I consider an Excuse to be as general as an object, and is structurally the same as an object, in the general sense but that it has the added semantics of representing that normal data is not here, and the Excuse is named after the fact it gives a specific reason WHY normal data is missing.

As I've mentioned on occasion, my in progress work https://github.com/muldis/Muldis_Object_Notation/blob/master/spec/Muldis_Object_Notation.md can be informative here, which defines a structured type system of sorts including Excuse. Whatever type systems Perl has or is growing including Corinna or the stronger scalar types, my work is designed to map to it elegantly and can be used for serialization/deserialization/interchange/etc of any Perl values losslessly and keeping semantics.

@HaraldJoerg
Copy link

I totally like using objects for exceptions. Actually, this has been around for some time: die can pass an object, and a surrounding eval block can pick it up and use the object's properties. So what is new in this proposal? Should Perl core provide a common base class for exception objects? I actually doubt that there's much value in that, given how different Perl applications are.

In a rather large (Java) application I've been working on, we could collect a lot of experience with exceptions and it doesn't fit well with the example class at the beginning of this gist. Here are some things I noticed:

  • The method build_stack_trace is private? This means it is unconditionally called and can't be overridden by a subclass? Bad idea. In my experience stack traces are rarely useful. What you often would like to know are the actual parameters used in a call. This makes some sense in a language like C, but less so in Perl. If parameters are objects, then you can't inspect them (Dios), or at least not easily (Object::Pad). And if you can, you get tons of context, mostly useless. It took developers some time to realize that users of a web application don't appreciate looking at stack traces in their browser.
  • message is mandatory? If so, then the message string is part of the API and must not be changed. The text should be machine-readable, otherwise localization with e.g. Locale::Maketext won't work. If you change the text, then tests should break, to remind you about the consequences.
  • Java can't do that, but Perl can, and should: to_string should be overloaded for stringification. If an exception isn't caught where you expect it to be caught, this is an easy safeguard against the dreaded died: Exception(0x55b3956e92e8) in logfiles.

Some random remarks:

  • About Exceptions should be used for exceptions, not flow control. - It depends. If a condition can supply useful information about what went wrong, this can be transported in an exception object, without any need for a stack trace.
  • About Something like an 'autoexception' mechanism could be very useful - Perl has autodie. And guess what: It actually returns exception objects. So, you can distinguish between "file not found" and "permission denied" without parsing $! strings which depend on platform and language by looking at the numerical errno.
  • About FileException and their likes: This is, at best case, a category at the lowest level. I suggest that such exceptions are caught early, and the application should then provide the appropriate context. open doesn't know whether a failure is a user error or a bug in the software, but the application does. Localization of messages is relevant for user errors - but not for software bugs, and stack traces are relevant for software bugs - but not for user errors.

Ah, and because I've actually seen this one in the wild:

try {
    open my $fh, '<', $filename;
}

Subsequent reads will read from whatever $fh was in scope before the try block. This is fun to figure out if there actually is such a variable!

@duncand
Copy link

duncand commented Mar 4, 2022

@HaraldJoerg said:

  • It took developers some time to realize that users of a web application don't appreciate looking at stack traces in their browser.

Yes, but its much bigger than that. Lazy web app developers who simply print out the internal exception or error details on a web page are opening themselves up for huge security/privacy/experience issues. You should NEVER (except in private testing environments) be displaying any details of your inner workings to web users. Instead they should be getting a nondescript message that something was wrong, and the details only logged internally. The only times users should be getting details is when the problem is the user's fault, and then it should only be specifically relevant to what they did wrong, not how the service internally failed. So whether exceptions provide stack traces or anything else have nothing to do with browsers.

@Ovid
Copy link
Author

Ovid commented Mar 4, 2022

@HaraldJoerg @duncand Thank you for the great feedback!

@merrilymeredith
Copy link

I'd really appreciate exception objects in core, and core exceptions being objects even more. I think, much like the object system itself, there's an advantage to having a few key interfaces blessed to be part of core rather than let the library ecosystem have several different takes on it. And just as you mentioned the possibility of localized messages, getting this first step in opens the door to applications customizing to their needs just by changing the top of the hierarchy, applying behaviors through Roles, or what-have-you. To do it today with core string exceptions means you have fiddle with things when they're caught every time.

For more hierarchy inspo from a dynamic language rather than static, you might look to Ruby.

Something I'd keep in mind is an interface for an exception that wraps another - forr creating a wrapper, reading the wrapped value, and the wrapper's behavior with regard to error type matching. That's useful to contextualize errors beyond just what the stack trace says, such as a higher-level error (say, CreateBackupError) that wraps the low-level cause (NoSpaceError). It's maybe not super important but I see it as another case where it's good to have a blessed interface: so you don't have several styles of the same behavior where the accessor is named "cause" vs "wrapped" vs "inner" and everyone has to be defensive about which they happen to be catching at the time. Ruby has "cause" and it actually tries to capture the cause for you, while Go (≥ 1.13) uses "Unwrap()" and an inner error must be wrapped explicitly.

@daotoad
Copy link

daotoad commented Mar 9, 2022

Thanks for putting this together. Exception handling has been in need of some TLC for quite a while.

One of my major peeves with most logging systems is the notion that log levels should be an ordered set, with level N implying that all levels 1 through N are enabled. Treating log levels as mutually independent gives much more control when adjusting the verbosity of instrumentation.

Is trace greater or less than debug? Will you always want to log errors when you want to log warnings?

A few years ago I worked on a large code base that was in dire need of instrumentation to address all manner of Heisenbugs and other undesirable behavior. We were able to bring things under control by using a logging system that allowed us to toggle 7 different log level bits over lexical scopes. This strategy worked very well and helped us focus our testing efforts improved our system monitoring.

To me it feels like there are a lot of unquestioned assumptions built into how we all think about logging that we would do well to root out.

Baking in a hard dependency on log levels as being inherently ordered would enforce a particular, restrictive set of capabilities on anything that uses it.

@duncand
Copy link

duncand commented Mar 9, 2022

I completely agree with @daotoad . And that made me think of a great idea, something there is a LOT of precedent for already. Don't think of logging as levels, instead think that the system provides a set of named channels, some code sends messages to those channels by name, other code reads messages from those channels by name. Have distinct channels for whatever level of granularity of subject matter one might want to log about. So for example, channels named info and error and warning and whatever. Perl defines some channel names that it uses to log things to, users can choose to log to the same channels or new ones that they declare. Loggers are set to read from whatever channels the user wants and output it to files/console/etc. In the general case, messages are objects. See the Postgres feature NOTIFY for example https://www.postgresql.org/docs/13/sql-notify.html of precedent.

@duncand
Copy link

duncand commented Mar 9, 2022

Addendum, or similar to with exception objects, you could choose to have one notification channel, and messages are objects of some hiararchy, like exceptions have a hierarchy. Rather than separate named channels, there's one channel and readers can just filter on what message classes they are interested in, either individually or on any point in the hierarchy to include subtypes etc.

@duncand
Copy link

duncand commented Mar 9, 2022

Further to the last 2 points, Perl could be configured that whenever it throws an exception object it ALSO posts the same exception object to the notification queue. So exceptions and notifications are separate concepts but can easily be used together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment