Typing System

Discussion in 'CommandHelper' started by LadyCailin, Mar 5, 2013.

  1. LadyCailin

    LadyCailin Administrator Developer

    If you've been following the project very closely, you know that I'm in the middle of rewriting the compiler, to fix several deep bugs, and support many new features. One of the long term plans for MethodScript is for it to be a first class standalone language, usable in small and large projects, in large scale projects where a robust language is needed, as well as in small projects, where a rapid prototyping scripting language is used. In order to accomplish this, a typing system needs to be added. Currently, MethodScript is weakly typed, with a few compiler tricks to emulate strongly typed features where possible. This is not perfect though, because the compiler can only catch a small subset of programmer mistakes this way. For instance, take the following code:

    PHP:

    if(@userCondition){
        @output = 'string' + 'string'
    }
     
    Despite the fact that MethodScript is weakly typed, that particular code will trigger a compile error, that is, you don't have to actually run the code to figure out that it will break, you simply have to compile it. However, it is trivial to bypass this compiler magic by assigning a variable instead:

    PHP:

    @a = 'string'
    @b = 'string'
    @output = @a + @b # No longer a compile error, we'll have to wait until runtime to check this.
     
    In some cases, this is unavoidable, for instance, when dealing with user input, if they give us unacceptable input, by definition, we can't really handle that at compile time. But if it's the programmer's fault, there is no reason we shouldn't be able to handle this at compile time instead. To manage this, we can add strong typing.

    PHP:

    string @a = 'string'
    string @b = 'string'
    @output = @a + @b # Now it's a compiler error, because addition takes two numbers, not strings
     
    However, one of the advantages of scripting languages is that you don't need to write out the types for all variables, so for both backwards compatibility and in order to continue to support fast prototyping, types will not be required, unless you have specified that you are running in strict mode. The total changes are going to take a while to implement, and the eventual goal is to allow for object oriented coding, and allowing for custom user objects, but this will be added in several phases. To make you aware of what's coming, and how you can prepare your design process and coding standards to better take advantage of the features if you want them, or to avoid having to change old code if you don't, I want to outline the various phases that are coming.

    Phase 1
    Phase 1 will simply add built in type checks. This will only simplify the coding for the built in functions, and will not change your code at all. However, CastExceptions will be standardized, and will be done automatically for all functions, not just the select few that already have the compiler magic built in. A few functions with exceptionally difficult signatures have been simplified, but the compiler will immediately alert you if your code is incompatible, and you should be able to quickly fix all instances of them. This will however cause a few update breakages. The compiler itself is also being rewritten in this phase, which will correct a few long standing bugs in the compiler, and allow for quicker bug fixes in the future, should any bugs remain.

    Strict mode will also be introduced. Strict mode will allow for your code to error out in places that are normally simple warnings, and will enforce strong typing in the future. Strict mode won't do much in this phase, other than requiring that variables be initialized with something before they are used, requiring all strings to be quoted (removes bare string support) and ensuring that procs are defined before they are used.

    Semi-colons will eventually be required in strict mode, so in this phase, semi colons will be allowed, but optional, even in strict mode.

    PHP:

    @a = 2;
    @b = 2;
    @output = @a + @b;
     
    Arrays will be "duck typed" in this phase, that is they won't actually be objects, but they will work like one, however, this will be a temporary support system in place before objects become fully supported, and built in functions will start to use object-like structures in all cases. This will however simplify both the java code, and the documentation, so things like the "locationArray" will be well defined, with their own page of documentation, and in the case of constant arrays, will allow for some compiler magic to be used to catch errors. Generics will be specified in the documentation, but unusable in code.

    ALL undefined unquoted symbols will cause compile errors at this point, even in non strict mode, so code such as: msg(You & me) will be a compile error. Though it has been in the documentation since day 1 for user code to always quote all symbols, I have found that many people have ignored this, so unfortunately this may cause some major breakages in some people's code.

    Additionally, includes will be restricted to only allow non-executable code inside of them (so procs and other includes) as detailed in this post.

    Phase 1 is still in development, and will take several more months at minimum to complete, but I am actively working on it, and much of the hard work for future phases will have been completed at that point. Upon release, this will be version 3.4.0.

    Phase 2

    Phase 2 will add actual object support, for built in objects only. This will introduce new syntax for accessing object members and methods, as well as constructor syntax, annotations, enums, and masks, and allow for variables to be typed. Strict mode now require semi-colons for statements, as closures and anonymous methods will be executable using new syntax, and the non-strict mode behaviour would be ambiguous. Strict mode will not require types at this point, but a warning will be issued if the type is missing, and strict mode is on, for both variables, and procs. Many of the built in functions will start providing both procedural and object oriented methods, and new classes of functions will probably only support object oriented versions. Generics will not be enforced by the compiler, but will be available for use in user code, and will eventually be enforced in strict mode.

    Existing code in non strict mode isn't expected to have any major breaking changes.

    This phase will take a few months at minimum. Upon release, this will be version 3.5.0.

    Phase 3

    Phase 3 will add the ability for user objects to be created. Strict mode will be required to use this feature though. In strict mode, all variables and procedures must be explicitly typed, or an error will be issued at this point.

    Existing code in non strict mode isn't expected to have any major breaking changes.

    This phase will take a few months at minimum. Upon release, this will be version 4.0.0.


    Other features will still continue to be added in the meantime, (there are 3 main sections of code, core language, core API, Minecraft API) but these are just the plans for the core language. After these 3 phases are implemented, the next task for the core language support will be for an IDE plugin (likely for NetBeans) to be implemented. This will simplify development greatly. I currently have a few people helping to add more minecraft functions while I work on this, but rest assured, I'm still hard at work on the project :)
    MariuszT likes this.
  2. ray73864

    ray73864 Member

    Looking forward to this.

    I love the idea of the strict mode, while it is handy being able to not have to specify the variable/object type, i like the idea that i can choose whether i want to enforce type-checks or not.

    The whole object-oriented side of it i think will be a plus too, finally code will be able to better behave, you could have 3 programs that all include the same base object and that base object defines all its functions and procs. Where as currently, if you include a file, the way you access those procs and funcs isn't defined, you can just do myproc(stuff), heaven forbid you have 2 includes that have that same proc called 'myproc', where as with objects, you can have 2 objects with the same functions and procs and just do 'myobj.myproc(stuff)'.

    I assume later on down the track there will be the ability for us to access the public methods of other plugins? things like Vault, TagAPI, Sync, etc... That would be a major plus, that way you don't have to code everything into CH, instead, if a person wants SQL support, they could just install the Sync plugin and access its public methods from their CH script.
  3. LadyCailin

    LadyCailin Administrator Developer

    If you are just dying to read more detail about some of these features, the following (raw) documentation may be interesting to you:

    https://github.com/sk89q/commandhelper/blob/NewCompiler/src/main/resources/docs/Cross_Casting
    https://github.com/sk89q/commandhelper/blob/NewCompiler/src/main/resources/docs/FileOptions
    https://github.com/sk89q/commandhelper/blob/NewCompiler/src/main/resources/docs/NewObjects
    https://github.com/sk89q/commandhelper/blob/NewCompiler/src/main/resources/docs/BasicBuiltinTypes
    https://github.com/sk89q/commandhelper/blob/NewCompiler/src/main/resources/docs/Annotations
    https://github.com/sk89q/commandhelper/blob/NewCompiler/src/main/resources/docs/Enums_and_Masks

    Read the article on cross casting, it goes into more detail about how that will work exactly. In non-strict mode, all untyped variables will be considered as if they were typed as "auto", and even in strict mode you can take advantage of this by typing something to auto. (Unrelated to C++'s auto keyword.)

    Actually, probably not directly, no. However, there is already in place an *unsupported* extension system that several people are using. There is a site that hosts builds of several of these extensions: http://build.zeoldcraft.com/ and source code for CHAdvanced can be found at https://github.com/LadyCailin/CHAdvanced. The reason I'm not officially supporting these yet is because the fundamentals of CH itself are still undergoing pretty big changes, and I don't want to have to support backwards compatibility for features that are subject to changing. Having said that, extensions are meant to wrap external java libraries (or in the case of a few of these extensions, other plugins). SQL will be added directly to CH though, so I would discourage anyone from adding that as an extension.

    Waay down the road, I want for MethodScript to be a good replacement of PHP, javascript, python, bash, and many other languages, so that will require several built in utilities to make usable. Already, MethodScript can be run from the command line, though it's usefulness is fairly limited due to it's lack of command line focused functions.
    Last edited: Sep 17, 2014
  4. ethancedrik

    ethancedrik New Member

    Will this cause poorly written code to break?
    Also CHAdvanced looks pretty cool!
    MethodScript would be a good replacement
  5. surfarcher

    surfarcher Member

    Awesome work @LadyCailin.

    @ethancedrik more likely poor code simply won't compile under CHAdvanced.
  6. LadyCailin

    LadyCailin Administrator Developer

    It depends on your definition of "poorly written." For the most part, if you've followed all the rules laid out in the docs, your code should be fine. Having said that, some of the rules were stated, but not enforced, for instance, the "all symbols must be quoted" rule. Many people have used, for instance
    Code (Text):
    runas(~console, '/blah')
    instead of quoting the '~console', and so that will break, but the rules have ALWAYS stated that symbols must be quoted, even though previously they worked as is. As much as is possible though, I'm going to try to do warnings in the first builds, then errors later, but that's a lot of work for me to do, so it will be limited.

    CHAdvanced is not a compiler extension, it just adds new events/functions.
  7. surfarcher

    surfarcher Member

    Apologies, I was more caught up in post 1, strong typing and similar... Which I am very much looking forward too. I kinda skipped over the unsupported extra plugin and confused it with the CH future direction.
  8. dral

    dral New Member

    I would love to see objects appear in commandhelper!