Add runtime encoding detection and conversionfix/char-detection

	Win-1251/1252 detection is a reduced C++ version of https://github.com/hsivonen/chardetng
Add manually-specified encoding fallback
Add default system encoding fallback
Add error recovery to v2script
Add unknown encoding detection warning
Remove csv::Parser templating
Fix lua files dropping data

Update lexy to foonathan/lexy@1e5d99fa3826b1c3c8628d3a11117fb4fb4cc0d0

Remove exclusive reliance on lexy::default_encoding for v2script

Move internal concepts to src/openvic-detail/InternalConcepts.hpp
Move contents of DetectUtf8.hpp to src/detail/Detect.hpp
Move openvic-dataloader/AbstractSyntaxTree.hpp to src
Move DiagnosticLogger.hpp to src
Move File.hpp to src
Move openvic-dataloader/detail/utlity files to openvic-dataloader/detail

Add ovdl::utility::type_concat
Add ovdl::utility::type_prepend
Add ovdl::utility::is_instance_of
Overhaul parse error messages

1 files changed, 11 insertions, 1 deletions

diff --git a/src/openvic-dataloader/File.cpp b/src/openvic-dataloader/File.cpp
index 9b27bf0..e4d3773 100644
--- a/src/openvic-dataloader/File.cpp
+++ b/src/openvic-dataloader/File.cpp
@@ -1,4 +1,10 @@
-#include <openvic-dataloader/File.hpp>
+#include "File.hpp"
+
+#include <cstring>
+
+#include <openvic-dataloader/detail/Utility.hpp>
+
+#include <lexy/encoding.hpp>
 
 using namespace ovdl;
 
@@ -6,4 +12,8 @@ File::File(const char* path) : _path(path) {}
 
 const char* File::path() const noexcept {
 	return _path;
+}
+
+bool File::is_valid() const noexcept {
+	return _buffer.index() != 0 && !_buffer.valueless_by_exception() && visit_buffer([](auto&& buffer) { return buffer.data() != nullptr; });
 }
\ No newline at end of file