Lua crashes when spawning entities in optimized builds #19

New Issue

Closed

opened 2024-06-16 20:39:13 +00:00 by SeanOMik · 1 comment

SeanOMik commented

2024-06-16 20:39:13 +00:00

Owner

When Lua spawns an entity, the engine crashes with a SIGABRT error (maybe SIGSEGV). This crash happens when spawning an entity on a world and passing a Transform, causing the userdata's __lyra_internal_reflect function to be called which clones this.0 (src).

I have no clue what causes it yet, its likely something in the elua crate. I tried to make a test to figure it out, but I couldn't get the crash to trigger

#[cfg(test)]
mod tests {
    use crate::{tests::Vec2, AnyUserdata, FromLua, State, StdLibraries, Userdata};

    #[derive(Copy, Clone, Debug)]
    struct InnerWrapper(Vec2);

    impl<'a> FromLua<'a> for InnerWrapper {
        fn from_lua(_: &'a State, val: crate::Value<'a>) -> crate::Result<Self> {
            let ud = val.as_userdata().unwrap();
            Ok(ud.as_ref::<Self>()?.clone())
        }
    }

    impl Userdata for InnerWrapper {
        fn name() -> String {
            "Vec2".into()
        }

        fn build<'a>(builder: &mut crate::UserdataBuilder<'a, Self>) {
            builder.function("new", |_, (x, y)| {
                Ok(Self(Vec2::new(x, y)))
            });
        }
    }

    struct Wrapper(InnerWrapper);

    impl Userdata for Wrapper {
        fn name() -> String {
            "Vec2Wrapper".into()
        }

        fn build<'a>(builder: &mut crate::UserdataBuilder<'a, Self>) {
            builder.function("new", |_, (v2_wrap,): (InnerWrapper,)| {
                Ok(Wrapper(v2_wrap))
            });

            builder.method("test_clone", |_, this, ()| {
                let inner = this.0.clone();
                println!("Inner: {:?}", inner);
                let inner_opt = Some(inner);
                println!("Inner option: {:?}", inner_opt);

                Ok(())
            });
        }
    }

    struct DoWorker;

    impl Userdata for DoWorker {
        fn name() -> String {
            "Worker".into()
        }

        fn build<'a>(builder: &mut crate::UserdataBuilder<'a, Self>) {
            builder.method("call_test_clone", |_, _, ud: AnyUserdata| {
                ud.execute_method::<_, ()>("test_clone", ())?;

                Ok(())
            });
        }
    }

    #[test]
    fn cloning_wrapper() -> crate::Result<()> {
        let script = r#"
            local inner = Vec2.new(50, 50)
            print("creating wrapper")
            local wrapped = Vec2Wrap.new(inner)
            
            print("now testing clone")
            worker:call_test_clone(wrapped)
        "#;

        let lua = State::new();
        lua.expose_libraries(StdLibraries::all());

        let globals = lua.globals()?;
        globals.set("Vec2", lua.create_proxy::<InnerWrapper>()?)?;
        globals.set("Vec2Wrap", lua.create_proxy::<Wrapper>()?)?;
        globals.set("worker", lua.create_userdata(DoWorker)?)?;

        lua.load("test", script)?.execute::<_, ()>(())?;

        Ok(())
    }
}

When Lua spawns an entity, the engine crashes with a SIGABRT error (maybe SIGSEGV). This crash happens when spawning an entity on a world and passing a `Transform`, causing the userdata's `__lyra_internal_reflect` function to be called which clones `this.0` ([src](https://git.seanomik.net/SeanOMik/lyra-engine/src/branch/debug/sig-abort/lyra-scripting/src/lua/wrappers/math.rs#L333)). I have no clue what causes it yet, its likely something in the [elua](https://git.seanomik.net/SeanOMik/elua/) crate. I tried to make a test to figure it out, but I couldn't get the crash to trigger ```rust #[cfg(test)] mod tests { use crate::{tests::Vec2, AnyUserdata, FromLua, State, StdLibraries, Userdata}; #[derive(Copy, Clone, Debug)] struct InnerWrapper(Vec2); impl<'a> FromLua<'a> for InnerWrapper { fn from_lua(_: &'a State, val: crate::Value<'a>) -> crate::Result<Self> { let ud = val.as_userdata().unwrap(); Ok(ud.as_ref::<Self>()?.clone()) } } impl Userdata for InnerWrapper { fn name() -> String { "Vec2".into() } fn build<'a>(builder: &mut crate::UserdataBuilder<'a, Self>) { builder.function("new", |_, (x, y)| { Ok(Self(Vec2::new(x, y))) }); } } struct Wrapper(InnerWrapper); impl Userdata for Wrapper { fn name() -> String { "Vec2Wrapper".into() } fn build<'a>(builder: &mut crate::UserdataBuilder<'a, Self>) { builder.function("new", |_, (v2_wrap,): (InnerWrapper,)| { Ok(Wrapper(v2_wrap)) }); builder.method("test_clone", |_, this, ()| { let inner = this.0.clone(); println!("Inner: {:?}", inner); let inner_opt = Some(inner); println!("Inner option: {:?}", inner_opt); Ok(()) }); } } struct DoWorker; impl Userdata for DoWorker { fn name() -> String { "Worker".into() } fn build<'a>(builder: &mut crate::UserdataBuilder<'a, Self>) { builder.method("call_test_clone", |_, _, ud: AnyUserdata| { ud.execute_method::<_, ()>("test_clone", ())?; Ok(()) }); } } #[test] fn cloning_wrapper() -> crate::Result<()> { let script = r#" local inner = Vec2.new(50, 50) print("creating wrapper") local wrapped = Vec2Wrap.new(inner) print("now testing clone") worker:call_test_clone(wrapped) "#; let lua = State::new(); lua.expose_libraries(StdLibraries::all()); let globals = lua.globals()?; globals.set("Vec2", lua.create_proxy::<InnerWrapper>()?)?; globals.set("Vec2Wrap", lua.create_proxy::<Wrapper>()?)?; globals.set("worker", lua.create_userdata(DoWorker)?)?; lua.load("test", script)?.execute::<_, ()>(())?; Ok(()) } } ```

SeanOMik added the

Priority

High

Kind/Bug

labels 2024-06-16 20:39:13 +00:00

SeanOMik changed title from ~~Lua crashes when spawning entities with optimizer or release mode.~~ to Lua crashes when spawning entities in optimized builds

2024-06-22 19:46:49 +00:00

SeanOMik added reference fix/scripting-switch-to-mlua

2024-09-30 01:29:32 +00:00

SeanOMik commented

2024-09-30 01:34:00 +00:00

Author

Owner

I ended up fixing this by switching to mlua for the lua library. It was fun writing my own lua library, elua, but there's so much unsafe code in it, I would prefer to use someone who knows more about what they're doing lol.

I ended up fixing this by switching to mlua for the lua library. It was fun writing my own lua library, elua, but there's so much `unsafe` code in it, I would prefer to use someone who knows more about what they're doing lol.

SeanOMik closed this issue

2024-09-30 01:34:01 +00:00

SeanOMik referenced a pull request that will close this issue

2024-09-30 01:38:32 +00:00

Fix #19: Lua crashes when spawning entities in optimized builds #27

SeanOMik referenced this issue from a commit

2024-09-30 01:39:42 +00:00

Merge pull request 'Fix #19: Lua crashes when spawning entities in optimized builds' (#27) from fix/scripting-switch-to-mlua into main