Error Handling

This guide covers common errors and issues you may encounter when using UnDatasIO, along with solutions and best practices for handling them effectively.

Common Error Types

File Upload Errors

File Too Large

Error Message: File size exceeds maximum limit of 100MB

Causes:

File size exceeds platform limits
Network timeout during upload
Server configuration limits

Solutions:

// Check file size before upload
const file = document.getElementById("fileInput").files[0];
if (file.size > 100 * 1024 * 1024) {
  // 100MB
  console.log("File too large, consider compression or splitting");
  // Compress file or split into smaller chunks
}

// Use streaming upload for large files
const stream = client.createUploadStream("large-file.csv");
stream.on("progress", (progress) => {
  console.log(`Upload progress: ${progress}%`);
});

Unsupported File Format

Error Message: Unsupported file format: .xyz

Causes:

File extension doesn’t match actual content
File is corrupted or empty
Format not supported by UnDatasIO

Solutions:

// Check supported formats
const supportedFormats = ["csv", "json", "xlsx", "xls", "txt", "xml", "pdf"];

// Validate file format
function validateFileFormat(file) {
  const extension = file.name.split(".").pop().toLowerCase();
  if (!supportedFormats.includes(extension)) {
    throw new Error(`Unsupported format: ${extension}`);
  }
  return true;
}

// Try manual format selection
const uploadResult = await client.uploadFile(file, {
  format: "csv", // Force format detection
  autoDetect: false,
});

Network Errors

Error Message: Network error: Failed to upload file

Causes:

Unstable internet connection
Server downtime
Firewall blocking uploads

Solutions:

// Implement retry logic
async function uploadWithRetry(file, maxRetries = 3) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await client.uploadFile(file);
    } catch (error) {
      if (i === maxRetries - 1) throw error;
      console.log(`Upload failed, retrying... (${i + 1}/${maxRetries})`);
      await new Promise((resolve) => setTimeout(resolve, 1000 * (i + 1)));
    }
  }
}

Parsing Errors

Encoding Issues

Error Message: Encoding error: Unable to decode file content

Causes:

File uses non-standard encoding
Mixed encodings in file
Corrupted characters

Solutions:

// Try different encodings
const encodings = ["UTF-8", "ISO-8859-1", "Windows-1252", "ASCII"];

for (const encoding of encodings) {
  try {
    const result = await client.parseFile(fileId, {
      encoding: encoding,
      format: "csv",
    });
    console.log(`Success with encoding: ${encoding}`);
    break;
  } catch (error) {
    console.log(`Failed with encoding: ${encoding}`);
  }
}

Delimiter Detection Issues

Error Message: Parsing error: Unable to detect delimiter

Causes:

Inconsistent delimiters in file
Mixed delimiters
Complex data structure

Solutions:

// Manual delimiter detection
const delimiters = [",", ";", "\t", "|", "|"];

for (const delimiter of delimiters) {
  try {
    const result = await client.parseFile(fileId, {
      delimiter: delimiter,
      hasHeader: true,
    });
    console.log(`Success with delimiter: ${delimiter}`);
    break;
  } catch (error) {
    console.log(`Failed with delimiter: ${delimiter}`);
  }
}

Schema Validation Errors

Error Message: Schema validation failed: Invalid data type

Causes:

Data doesn’t match expected schema
Missing required fields
Invalid data types

Solutions:

// Flexible schema validation
const flexibleSchema = {
  type: "object",
  properties: {
    id: { type: ["integer", "string"] }, // Accept both types
    name: { type: "string" },
    email: { type: "string", format: "email" },
  },
  required: ["name"], // Only name is required
  additionalProperties: true, // Allow additional properties
};

const result = await client.parseFile(fileId, {
  schema: flexibleSchema,
  strict: false, // Don't fail on validation errors
});

Processing Errors

Memory Issues

Error Message: Memory error: Insufficient memory for processing

Causes:

Dataset too large for available memory
Inefficient processing operations
Memory leaks in processing pipeline

Solutions:

// Use streaming processing
const streamProcessor = client.createStreamProcessor({
  batchSize: 1000,
  maxMemory: "1GB",
});

// Process data in chunks
const chunks = await client.splitData(data, { chunkSize: 10000 });
for (const chunk of chunks) {
  const processed = await client.processData(chunk, operations);
  // Handle processed chunk
}

Transformation Errors

Error Message: Transformation error: Invalid operation

Causes:

Invalid transformation parameters
Data type mismatches
Unsupported operations

Solutions:

// Validate transformation parameters
function validateTransformation(operation) {
  const validOperations = ["uppercase", "lowercase", "trim", "replace"];
  if (!validOperations.includes(operation.type)) {
    throw new Error(`Invalid operation: ${operation.type}`);
  }

  // Check column exists
  if (!data.columns.includes(operation.column)) {
    throw new Error(`Column not found: ${operation.column}`);
  }
}

// Safe transformation with error handling
try {
  const result = await client.transformData(data, operations);
} catch (error) {
  console.log("Transformation failed:", error.message);
  // Fallback to basic processing
  const basicResult = await client.basicProcess(data);
}

API Errors

Authentication Errors

Error Message: 401 Unauthorized: Invalid API key

Causes:

Invalid or expired API key
Incorrect authentication method
Account suspended

Solutions:

// Validate API key
async function validateApiKey(apiKey) {
  try {
    const response = await fetch("https://api.undatasio.com/v1/auth/validate", {
      headers: { Authorization: `Bearer ${apiKey}` },
    });
    return response.ok;
  } catch (error) {
    return false;
  }
}

// Handle authentication errors
try {
  const result = await client.uploadFile(file);
} catch (error) {
  if (error.status === 401) {
    console.log("Authentication failed, check API key");
    // Prompt user to update API key
  }
}

Rate Limit Errors

Error Message: 429 Too Many Requests: Rate limit exceeded

Causes:

Too many API requests in short time
Exceeded plan limits
Concurrent requests limit

Solutions:

// Implement rate limiting
class RateLimiter {
  constructor(maxRequests = 100, timeWindow = 60000) {
    this.maxRequests = maxRequests;
    this.timeWindow = timeWindow;
    this.requests = [];
  }

  async waitForSlot() {
    const now = Date.now();
    this.requests = this.requests.filter(
      (time) => now - time < this.timeWindow
    );

    if (this.requests.length >= this.maxRequests) {
      const waitTime = this.timeWindow - (now - this.requests[0]);
      await new Promise((resolve) => setTimeout(resolve, waitTime));
    }

    this.requests.push(now);
  }
}

const rateLimiter = new RateLimiter(100, 60000); // 100 requests per minute

// Use rate limiter
async function apiCall() {
  await rateLimiter.waitForSlot();
  return await client.apiCall();
}

Server Errors

Error Message: 500 Internal Server Error

Causes:

Server maintenance
Database issues
Configuration problems

Solutions:

// Implement exponential backoff
async function apiCallWithBackoff(fn, maxRetries = 5) {
  for (let i = 0; i < maxRetries; i++) {
    try {
      return await fn();
    } catch (error) {
      if (error.status < 500 || i === maxRetries - 1) throw error;

      const delay = Math.pow(2, i) * 1000; // Exponential backoff
      console.log(`Server error, retrying in ${delay}ms...`);
      await new Promise((resolve) => setTimeout(resolve, delay));
    }
  }
}

Error Handling Best Practices

Comprehensive Error Handling

// Comprehensive error handler
class UnDatasIOErrorHandler {
  constructor(client) {
    this.client = client;
    this.retryAttempts = 3;
    this.retryDelay = 1000;
  }

  async handleUpload(file) {
    try {
      // Validate file
      this.validateFile(file);

      // Upload with retry
      return await this.uploadWithRetry(file);
    } catch (error) {
      this.logError("Upload failed", error);
      this.notifyUser(error);
      throw error;
    }
  }

  async handleParsing(fileId, options) {
    try {
      return await this.client.parseFile(fileId, options);
    } catch (error) {
      if (error.code === "ENCODING_ERROR") {
        return await this.tryDifferentEncodings(fileId, options);
      } else if (error.code === "DELIMITER_ERROR") {
        return await this.tryDifferentDelimiters(fileId, options);
      } else {
        throw error;
      }
    }
  }

  validateFile(file) {
    if (!file) throw new Error("No file provided");
    if (file.size > 100 * 1024 * 1024) throw new Error("File too large");

    const supportedFormats = ["csv", "json", "xlsx", "xls"];
    const extension = file.name.split(".").pop().toLowerCase();
    if (!supportedFormats.includes(extension)) {
      throw new Error(`Unsupported format: ${extension}`);
    }
  }

  async uploadWithRetry(file) {
    for (let i = 0; i < this.retryAttempts; i++) {
      try {
        return await this.client.uploadFile(file);
      } catch (error) {
        if (i === this.retryAttempts - 1) throw error;
        await this.delay(this.retryDelay * (i + 1));
      }
    }
  }

  async tryDifferentEncodings(fileId, options) {
    const encodings = ["UTF-8", "ISO-8859-1", "Windows-1252"];

    for (const encoding of encodings) {
      try {
        return await this.client.parseFile(fileId, {
          ...options,
          encoding: encoding,
        });
      } catch (error) {
        console.log(`Failed with encoding: ${encoding}`);
      }
    }
    throw new Error("All encodings failed");
  }

  async tryDifferentDelimiters(fileId, options) {
    const delimiters = [",", ";", "\t", "|"];

    for (const delimiter of delimiters) {
      try {
        return await this.client.parseFile(fileId, {
          ...options,
          delimiter: delimiter,
        });
      } catch (error) {
        console.log(`Failed with delimiter: ${delimiter}`);
      }
    }
    throw new Error("All delimiters failed");
  }

  logError(message, error) {
    console.error(`${message}:`, {
      message: error.message,
      code: error.code,
      status: error.status,
      timestamp: new Date().toISOString(),
    });
  }

  notifyUser(error) {
    // Show user-friendly error message
    const userMessage = this.getUserMessage(error);
    // Display in UI
  }

  getUserMessage(error) {
    const messages = {
      FILE_TOO_LARGE: "File is too large. Please compress or split the file.",
      UNSUPPORTED_FORMAT:
        "File format not supported. Please use CSV, JSON, or Excel files.",
      NETWORK_ERROR:
        "Network error. Please check your connection and try again.",
      AUTH_ERROR: "Authentication failed. Please check your API key.",
      RATE_LIMIT: "Too many requests. Please wait a moment and try again.",
    };

    return (
      messages[error.code] || "An unexpected error occurred. Please try again."
    );
  }

  delay(ms) {
    return new Promise((resolve) => setTimeout(resolve, ms));
  }
}

Error Monitoring and Logging

// Error monitoring service
class ErrorMonitor {
  constructor() {
    this.errors = [];
    this.maxErrors = 100;
  }

  logError(error, context = {}) {
    const errorLog = {
      timestamp: new Date().toISOString(),
      message: error.message,
      code: error.code,
      stack: error.stack,
      context: context,
    };

    this.errors.push(errorLog);

    // Keep only recent errors
    if (this.errors.length > this.maxErrors) {
      this.errors = this.errors.slice(-this.maxErrors);
    }

    // Send to monitoring service
    this.sendToMonitoring(errorLog);
  }

  async sendToMonitoring(errorLog) {
    try {
      await fetch("https://monitoring.undatasio.com/errors", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify(errorLog),
      });
    } catch (error) {
      console.error("Failed to send error to monitoring:", error);
    }
  }

  getErrorSummary() {
    const summary = {};
    this.errors.forEach((error) => {
      const code = error.code || "UNKNOWN";
      summary[code] = (summary[code] || 0) + 1;
    });
    return summary;
  }
}

Debugging Tools

Debug Mode

// Enable debug mode
const client = new UnDatasIO({
  apiKey: "your_api_key",
  debug: true,
});

// Debug logging
client.on("debug", (message) => {
  console.log("Debug:", message);
});

Request/Response Logging

// Log all API requests and responses
client.on("request", (request) => {
  console.log("Request:", {
    method: request.method,
    url: request.url,
    headers: request.headers,
    body: request.body,
  });
});

client.on("response", (response) => {
  console.log("Response:", {
    status: response.status,
    headers: response.headers,
    body: response.body,
  });
});

Getting Help

Self-Service Resources

Documentation: Check the relevant documentation sections
FAQ: Review frequently asked questions
Community Forum: Search for similar issues
Status Page: Check service status

Contacting Support

When contacting support, provide:

Error details: Full error message and code
Context: What you were trying to do
Steps to reproduce: How to recreate the issue
Environment: Browser, OS, SDK version
Logs: Any relevant error logs

Error Reporting Template

// Error reporting template
function reportError(error, context) {
  const report = {
    timestamp: new Date().toISOString(),
    error: {
      message: error.message,
      code: error.code,
      status: error.status,
      stack: error.stack,
    },
    context: {
      action: context.action,
      fileType: context.fileType,
      fileSize: context.fileSize,
      browser: navigator.userAgent,
      sdkVersion: client.version,
    },
    steps: context.steps || [],
  };

  // Send to support
  return fetch("https://support.undatasio.com/report", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify(report),
  });
}

Prevention Strategies

Proactive Error Prevention

Validate inputs before processing
Use appropriate file formats for your data
Implement proper error handling in your code
Monitor system resources during processing
Test with sample data before processing large files

Regular Maintenance

Update SDK versions regularly
Monitor error logs for patterns
Review and optimize processing pipelines
Backup important data before processing
Test error recovery procedures

For more information on specific errors, check out: